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ABSTRACT 


Solution  to  the  problem  of  estimating  the  probability  that  a  giv¬ 
en  electronic  malfunction  will  be  corrected  within  a  given  time  is  ad¬ 
dressed  through  compatible,  complementary  techniques.  One  technique 
compounds  Fleet  derived  job  factor  success  probability  data  to  yield  a 
probability  of  malfunction  correction  success.  The  complementary 
technique,  based  on  computer  simulation  methods,  yields  the  antici¬ 
pated  time  for  malfunction  correction.  The  two  techniques  were  ap¬ 
plied  to  two  different  operational  Navy  systems.  The  results  are  pre¬ 
sented  and  discussed  in  the  context  of  technique  reliability,  utility, 
discriminating  power,  and  reasonableness  of  obtained  results. 


iii 


ACKNOWLEDGMENTS 


We  are  indebted  to  a  number  of  persons  and  agencies  for  their 
assistance  in  various  aspects  of  this  research  project.  James  Jenkins 
and  Robert  Howard  of  the  Naval  Ship  Systems  Command  coordinated 
the  program  and  provided  needed  assistance  throughout.  Joseph 
Blenkenheim  of  the  Naval  Electronic  Systems  Command  helped  pro¬ 
vide  necessary  background  information  regarding  the  first  electronic 
system  included  in  the  study.  LCDR  Joseph  Funaro  of  the  Naval  Air 
Systems  Command  is  acknowledged  for  his  help  in  identifying  the  sec¬ 
ond  electronic  system  appropriate  for  inclusion  in  this  study  and  for 
his  role  in  completing  the  necessary  field  arrangements. 

Our  appreciation  is  expressed  to  Donald  Peek  of  the  Naval 
Ships  Engineering  Center,  Norfolk  Division,  for  his  assistance  with 
the  task  analysis  of  the  AN/URC-35  radio  set,  and  to  ATI  Frank 
Trafford,  Naval  Air  Maintenance  Training  Detachment,  NAS  Patuxent 
River,  for  his  assistance  with  the  task  analysis  of  the  AN/APS-H5 
radar.  A  number  of  other  persons  at  NAS  Patuxent  River  also  helped 
in  coordinating  our  research  efforts  and  provided  important  informa¬ 
tion.  These  include,  but  are  not  limited  to;  CDR  A.  Woolery,  Donald 
Neal,  ATC  Roger  Montfort,  and  AFCM  B.  R.  Smith.  Four  electronic 
repairmen  were  interviewed  at  che  U.  S.  Naval  Station,  Annapolis,  for 
the  purpose  of  reviewing  and  critiquing  the  task  analysis  developed  on 
the  AN/URC-3b.  They  were  CPO  Feldbinger,  CPO  Geisler,  first  class 
petty  officer  Lowes,  and  second  class  petty  officer  Tyson. 

At  Applied  Psychological  Services,  W.  Rick  Leahy  program¬ 
med  the  data  for  computer  simulation  and  Douglas  H.  Macpherson  and 
Janet  L.  Siegel  assisted  with  the  data  reduction  and  analysis. 


IV 


TABLE  OF  CONTENTS 


FOREWORD .  i 

ABSTRACT  . iii 

ACKNOWLEDGMENTS . iv 

CHAPTER  I  -  INTRODUCTION .  1 

Overall  Logic  and  Event  Flow .  3 

Multidimensional  Scaling .  5 

Reliability  of  Maintenance  Technicians  on 

Each  Factor . 11 

Method  for  Compounding . 15 

Job  Activity  Repetition  . 22 

Computer  Simulation  . 23 

Overview  of  the  Model . 28 

CHAPTER  H  -  METHODS  AND  PROCEDURES . 41 

EQUIPMENT  DESCRIPTION  AND  SAMPLE  SELECTION . 42 

Description  of  Radio  Set  AN/URC-35 . 42 

Selection  of  Failed  Subassemblies  and 

Components--AN/URC-35 . 43 

Description  of  Radar  Set  AN/APS-115 . 45 

Selection  of  Failures- -AN/ APS- 115 . 47 

AUGMENTATION  OF  COMPUTER  SIMULATION  MODEL . 50 

Input  Data  Required . 51 

Flow  Logic  Description . 53 

Recorded  Preprocessor  Output . 59 

Frequency  Distribution  Augmentation . 59 

TECHNIQUE  APPLICATION . 62 

Task  Analyses  . 62 

Probability  of  Successful  Malfunction 

Correction  Determination . 64 

Time  for  Malfunction  Correction  Determination. ...  64 
Interuser  Reliability . 66 


v 


CHAPTER  III  -  RESULTS . 67 

Reliability  Estimations  for  the  AN/ APS- 115  69 

TIME  B'OR  MALFUNCTION  CORRECTION . 71 

Series  Case--AN/ URC-35 . 71 

Parallel  Case--AN/URC-35 . 76 

Series  Case--AN/APS-115 . 78 

Parallel  Case--AN/ APS-115 . 81 

Interuser  Reliability .  83 

CHAPTER  IV  -  DISCUSSION  AND  CONCLUSIONS . 85 

Conclusions .  88 

REFERENCES  .  89 

APPENDIX  A  . 93 


TABLE  OF  FIGURES 


Figure  Page 

1  Flow  of  events  in  maintenance  technician  reliability 

program . . .  4 

2  Mean  personnel  reliability  values  of  technical  ratings 

across  squadrons  on  maintenance  job  activities. . .  14 

3  Card  deck  layout .  35 

4  Summary  flow  chart  of  preprocessor .  54 

5  Basic  programming  elements  for  preprocessor .  55 

6  Example  of  preprocessing  calculation .  58 

7  Example  of  initial  preprocessor  output .  60 

8  Sample  final  preprocessor  output .  61 

9  Predicted  malfunction  correction  time  (single  techni¬ 

cian)  for  malfunctions  2,  4,  6,  and  9  (AN/URC-35)  72 

10  Predicted  malfunction  correction  time  (single  techni- 

cian)for  malfunctions  10,11,  and  18  (AN/URC-35)  73 

11  Predicted  malfunction  correction  time  (single  techni- 

cian)for  malfunctions  24  and  25  (AN/URC-35)  ....  74 

12  Predicted  malfunction  correction  time  (single  techni- 

cian)for  malfunctions  101,  105,  and  110  (AN/ APS-115)  79 

13  Predicted  malfunction  correction  time  (single  techni- 

cian)for  malfunctions  114  and  117  (AN/ APS-115) . .  80 


LIST  OF  TABLES 


Table  Page 

1  Definition  of  Factors  . * .  8 

2  Summary  of  Reports  on  Man-Machine  Simulation . 25 

3  Summary  of  Addresses  on  Man-Machine  Simulation . 27 

4  Subtask  Input  Data  and  Card  Format . 30 

5  Parameter  Input  Data  and  Card  Format . 33 

6  General  Run  Information  and  Card  Format . 34 

7  Actual  ana  Sample  Frequency  Distributions  of  Mean  Time 

Between  Failures  for  the  Receiver-Transmitter  and  the 
Radio  Frequency  Amplifier  Units . 44 

8  Sample  of  Components  for  Technician  Reliability  Analysis  of 

AN/URC-35 . 46 

9  Frequency  Distribution  of  Mean  Time  Between  Removals  for 

the  AN/ APS- 115  and  for  Sample . 48 

10  Sample  of  Components  for  Technician  Reliability  Analysis  of 

AN/  APS-115 . 49 

11  Job  Activity  (Factor)  Group  (JAG)  Codes . 52 

12  Job  Operator  Type  (JOT)  Codes . . 52 

13  Sample  Task  Analysis  . 63 

14  Technician  Reliability  Predictions  for  Sample  of  Malfunction 

Corrections  for  the  Radio  Set  AN/URC-35 . 68 

15  Technician  Reliability  Predictions  for  Sample  of  Malfunction 

Corrections  for  the  Radar  Set  AN/APS-115 . 70 

16  Cumulative  Percentage  of  Corrections  within  Stated  Times 

for  the  AN/URC-35  (Parallel  Technicians) . 77 

17  Cumulative  Percentage  of  Corrections  within  Stated  Times 

for  the  AN/APS-115  (Parallel  Technicians) . 82 


18  Reliability  Coefficients  Among  Analysts  Using  the  Job  Factors.  84 

viii 


CHAPTER  I 


INTRODUCTION 


Techniques  and  methods  for  predicting  equipment  reliability, 
in  terms  of  mean  time  to  failure,  have  become  an  integral  part  of  the 
engineering  armamentorium.  However,  parallel  methods  pertaining 
to  human  reliability  (the  probability  of  successful  performance)  have 
not  evolved.  Yet,  total  system  reliability  determinations  are  some¬ 
what  incomplete  when  the  human  aspect  is  ignored.  Total  system  re¬ 
liability  is  viewed  as  some  function  of  both  equipment  and  technician 
reliability.  Since  techniques  for  determining  equipment  reliability  are 
available.  Applied  Psychological  Services,  Inc, ,  in  the  past  few  years, 
has  addressed  itself  to  the  problem  of  developing  techniques  for  deter¬ 
mining  technician  reliability. 

A  technique  for  determining  the  reliability  of  the  human  compo¬ 
nent  in  a  system  must,  if  it  is  to  be  useful,  incorporate  several  differ¬ 
ent  concepts  and  characteristics.  First,  the  technique  should  yield  a 
numerical  estimate  of  predicted  reliability.  Second,  the  predicted  hu¬ 
man  reliability  estimate  should  be  amenable  to  compounding  with  an 
equipment  reliability  estimate  so  as  to  provide  a  total  system  reliabil¬ 
ity  estimate.  This  requirement  indicates  that  the  human  reliability 
statement  should  be  expressed  as  a  probability. 

Third,  the  human  reliability  predictive  technique  should  be  able 
to  yield  not  only  a  total  system  or  task  reliability  statement,  but  also 
statements  of  why  a  given  human  reliability  determination  is  high  or 
low.  Since  an  equipment  designer  is  interested  in  learning  the  areas 
in  which  his  system  is  weak,  an  overall  human  reliability  numeric  rep¬ 
resents  a  necessary,  but  not  sufficient,  index.  The  human  reliability 
index  should  provide  statements  of  task  subsequence  reliability,  as  well 
as  a  total  sequence,  reliability  estimate.  In  hardware  reliability,  the 
counterparts  for  subsequences  and  total  sequences  are  subsystems  and 
systems. 
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Fourth,  the  technique  must  be  applicable  early  in  the  system 
development  cycle.  If  the  required  human  reliability  prediction  fails 
to  become  available  until  late  in  the  design  cycle,  the  cost  impact  of 
any  indicated  design  change  could  be  excessive. 

A  fifth  requirement  involves  technique  practicality.  Practi¬ 
cality  infers  cost  minimization  as  well  as  ease  of  application.  A  tech¬ 
nique  which  can  be  employed  by  a  minimally  trained  analyst  is  held  to 
be  more  practiced  than  one  which  implies  excessive  mathematical  or 
other  sophistication.  Similarly,  a  technique  which  is  compatible  with 
hand  calculational  or  desk  calculator  methods  is  considered  to  be  more 
practical  than  one  which  rests  on  the  availability  of  high  speed  digital 
computers. 

Sixth,  the  technique  must  be  applicable  to  a  wide  variety  of  sys¬ 
tems,  i.  e. ,  the  technique  must  possess  generality.  A  technique  which 
is  too  broadly  based  will  tend  to  lose  veridicality  for  any  specific  situ¬ 
ation.  On  the  other  hand,  a  technique  which  is  highly  specific  will  pos¬ 
sess  high  relevance  for  the  given  situation,  but  may  drastically  miss 
the  mark  when  applied  to  other  situations.  For  maximum  utility,  a 
technique  is  sought  which  optimizes  applicability  and  veridicality. 

Seventh,  the  technique  should  be  fully  compatible  with  specified 
end  products  that  emerge  from  human  factors  analyses,  currently  per¬ 
formed  during  system  development.  Moreover,  the  technique  should 
impose  few  analytic  requirements  other  than  those  imposed  by  actual 
technique  application.  More  specifically,  if  task  or  operational  sequence 
data  are  required  by  the  technique,  the  data  requirements  should  be  di¬ 
rectly  based  on  information  which  is  customarily  made  available  during 
the  equipment  developmental  cycle. 

Eighth,  the  technique  should  be  valid.  Validity  in  the  present 
sense  means  predictive  validity  as  well  as  content  and  construct  valid¬ 
ity.  Validity,  in  the  present  context,  also  relates  to  the  mathematical 
procedures  which  are  involved  and  to  the  reasonableness  of  the  math¬ 
ematical  assumptions. 

Ninth,  the  technique  should  be  psychometrically  reliable.  Dif¬ 
ferent  users  should  obtain  the  same  numeric  when  applying  the  tech¬ 
nique  to  the  same  system.  Additionally,  the  same  user  should  obtain 
the  same  numeric  when  applying  the  technique  to  the  same  system,  on 
different  occasions. 
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Finally,  the  human  reliability  technique  should  yield  a  state¬ 
ment  of  the  time  it  will  take  a  technician  to  complete  a  given  task,  as 
well  as  the  probability  of  satisfactory  task  completion. 

Overall  Logic  and  Event  Flow 

Our  overall  logic  for  the  development  of  a  technician  reliabil¬ 
ity  determination  technique  rests  on  two  approaches:  (1)  a  determin¬ 
istic  method  for  calculating  the  probability  that  an  average  maintenance 
technician  will  successfully  accomplish  the  sequence  of  acts  required 
for  malfunction  correction,  and  (2)  a  stochastic,  digital  simulation 
method  for  predicting  the  time  that  it  will  take  to  accomplish  the  cor¬ 
rection. 

Figure  1  presents  the  overall  flow  of  events  leading  to  and  in¬ 
cluded  in  the  present  effort.  The  heavy  lines  in  Figure  1  enclose 
those  steps  which  were  completed  prior  to  the  present  study  and  on 
which  the  present  study  attempted  to  build.  The  logic  on  which  the  de¬ 
terministic  calculation  is  based  calls  for  a  multidimensional  scaling 
analysis  to  yield  the  factors  (in  the  factor  analytic  sense)  involved  in 
electronic  maintenance.  Having  isolated  these  factors,  technician  re¬ 
liability  was  determined  on  each  of  the  factors.  In  the  present  context, 
technician  reliability  refers  to  an  estimate  of  the  probability  that  an 
"average"  maintenance  technician  will  perform  each  factor,  isolated 
in  the  multidimensional  scaling  analysis,  satisfactorily  when  the  fac¬ 
tor  is  involved  in  a  malfunction  correction.  Since  a  malfunction  cor¬ 
rection  involves  several  of  the  factors,  the  next  step  involved  the  de¬ 
velopment  of  a  method  for  compounding  technician  reliability  on  each 
factor  in  such  a  way  that  a  numeric  would  result  which  expressed  the 
probability  of  a  maintenance  technician  satisfactorily  performing  in 
combination  those  factors  involved  in  any  given  malfunction  correc¬ 
tion.  The  technique  was  then  applied  to  a  sample  of  malfunctions,  all 
representative  of  malfunction  corrections  in  the  Navy. 
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A  computer  simulation  model  was  previously  developed  which 
has  shown  considerable  promise  for  predicting  operator  performance 
in  man/ machine  systems.  The  model,  which  has  become  known  as  the 
Siegel-Wolf  simulation  model,  has  been  fully  described  elsewhere 
(Siegel  &  Wolf,  1969)  and  is  used,  in  the  present  context,  for  predict¬ 
ing  malfunction  correction  time.  For  the  purpose  of  employing  this 
model  in  a  manner  which  possesses  maximum  synergism  with  the  tech¬ 
nician  factor  reliability  data  employed  in  the  deterministic  calculation, 
certain  modifications  were  implemented  in  the  logic  of  the  model.  These 
modifications  were  implemented  through  a  preprocessing  technique. 

This  preprocessing  serves  to  enable  the  model  to  act  on  the  same  tech¬ 
nician  factor  reliability  data  employed  in  the  deterministic  calculation. 
The  advantage  of  the  preprocessing  technique  is  that  it  leaves  the  in¬ 
ternal  logic  of  the  model,  and  accordingly,  the  model's  validity,  un¬ 
changed.  The  end  result  is  a  prediction  of  time  to  complete  the  mal¬ 
function  correction  in  a  manner  that  is  entirely  compatible  with  the 
deterministic  calculation  of  the  probability  of  satisfactory  malfunction 
correction. 

The  details  and  rationale  for  each  of  the  steps  completed  earli¬ 
er  are  presented  in  subsequent  sections  of  Chapter  I  of  this  report. 
Subsequent  chapters  describe  the  tasks  completed  in  the  first  year  of 
the  current  program  of  work  toward  the  establishment  of  a  human  re¬ 
liability  predictive  technique  for  application  in  maintainability  predic¬ 
tion  situations. 


Multidimensional  Scalinf 


Multidimensional  scaling  analysis  is  a  comparatively  recent 
technique  for  defining  or  structuring  an  unordered  universe.  The 
technique  has  a  history  of  test  and  application  in  a  variety  of  contexts, 
including  work  on  attitudes  (Messick,  1954,  1956a;  Abelson,  1954), 
personality  (Jackson,  Messick,  &  Solley,  1957),  jobs  (Reeb,  1959), 
facial  expression  (Abelson,  1962),  civil  defense  (Smith  &  Siegel,  1967), 
and  display  evaluation  (Silver,  Landis,  &  Jones,  1965). 
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Originally  developed  by  Richardson  (1938),  multidimensional 
scaling  represents  an  expansion  of  factor  analysis  to  qualitative  do¬ 
mains.  Gullicksen  (1961),  in  summing  up  his  feelings  about  the  value 
of  multidimensional  scaling,  said  that  it: 

...  is  a  rather  powerful  technique  for  investigating 
a  wide  array  of  situations.  The  basic  experimen¬ 
tal  question  is  a  very  simple  one.  Despite  a  super¬ 
ficial  appearance  of  difficulty  and  unreasonableness, 
one  can  get  consistent  answers  and  can  come  up  with 
rather  interesting  conclusions- -some  of  which  veri¬ 
fy  the  results  of  unidimensional  scaling  and  others 
which  go  beyond  (p.  17). 

The  basic  logic  of  multidimensional  scaling  may  be  stated  as 
follows.  If  similarity  estimates  among  a  variety  of  stimuli  in  a  set 
can  be  assumed  to  be  measures  of  the  "psychological  distances"  be¬ 
tween  the  stimuli  in  Euclidean  space,  the  analytical  problem  becomes 
the  determination  of  the  number  of  axes  in  that  space  and  the  projec¬ 
tion  of  the  stimuli  on  these  axes.  Factor  analytic  methods  represent 
the  final  stage  of  a  multidimensional  scaling  analysis  and,  as  in  factor 
analysis,  the  result  is  a  statement  of  the  minimum  number  of  dimen¬ 
sions  (factors)  which  can  be  employed  to  describe  the  set  of  stimuli. 

The  scale  loadings  of  the  stimuli  on  each  dimension,  or  factor,  enable 
the  analyst  to  attach  meaning  to  and  so  to  name  the  dimensions. 

The  major  difference  between  multidimensional  scaling  and  uni¬ 
dimensional  scaling  is  that  in  the  typical  unidimensional  experiment  the 
dimensions  or  scales  are  presented  to  judges  whose  task  is  to  order  the 
stimuli  on  the  dimensions  defined  by  the  experimenter.  Conversely, 
there  are  no  such  a  priori  definitions  made  in  the  multidimensional 
scaling  experiment.  Rather,  the  purpose  of  the  analysis  is  to  discover 
the  characteristics  of  and  the  number  of  underlying  dimensions  in  the 
empirical  data. 

In  areas  where  the  variables  are  complex  and  the  dimensions 
unknown  or  doubtful,  it  is  appropriate  to  delineate  the  variables  through 
multidimensional  scaling  analysis,  rather  than  to  establish  the  dimen¬ 
sions  arbitrarily.  Evidence  of  the  validity  of  the  multidimensional  meth¬ 
od  has  been  cited,  especially  in  research  where  dimensions  are  well 
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established.  Messick,  in  his  studies  on  color,  concluded  that  "since 
multidimensional  scaling  procedures  yielded  structures  which  correlat¬ 
ed  highly  with  the  revised  Munsell  system,  it  would  now  seem  reason¬ 
able  to  apply  these  procedures  for  purposes  of  exploration  and  discov¬ 
ery  in  areas  of  unknown  dimensionality"  (1956c,  p.  374). 

To  obtain  the  dimensionality  of  the  job  of  the  electronic  main¬ 
tenance  technician,  Siegel  and  Schultz  (1963)  performed  a  multidimen¬ 
sional  scaling  analysis  of  the  job  of  Naval  technicians  concerned  with 
electronic  maintenance.  The  consensus  of  Fleet  personnel  interviewed 
indicated  that  the  job  of  the  technician  was  best  described  by  some  29 
different  tasks.  Examples  of  tasks  were:  "using  schematics  for  com¬ 
plex  circuits...  "  and  "troubleshooting/ isolating  malfunctions  in  avionic 
equipments.  "  The  29  tasks  constituted  the  stimuli  and  both  supervisory 
and  line  personnel  constituted  the  sample  who  provided  the  basic  inter¬ 
stimulus  distance  estimates  for  the  analysis.  Sixty-five  subjects  dis¬ 
tributed  over  14  separate  maintenance  units  were  involved.  The  cor¬ 
rected  matrix  of  scale  values  was  factor  analyzed  by  the  principal  com¬ 
ponents  method  with  rotation  according  to  the  equamax  criterion  (Saunders, 
1962).  Nine  factors  emerged.  These  factors  were  named:  Electro-cog¬ 
nition  (EC),  Electro-repair  (ER),  Instruction  (I),  Electro -safety  (ES), 
Personnel  Relationships  (PR),  Electronic  Circuit  Analysis  (ECA),  Equip¬ 
ment  Operation  (EO),  Using  Reference  Materials  (URM),  and  Equipment 
Inspection  (El).  These  factors  are  fully  defined  in  Table  1. 
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Table  1 


1.  Electro -cognition 


2,  Electro-repair 


3.  Instruction 


Definition  of  Factors 


This  factor  is  associated  with  any  routine  men¬ 
tal  activity  involved  in  the  troubleshooting  acts. 

It  includes  the  mental  formulation  of  simple  hy¬ 
potheses  regarding  the  cause  of  a  malfunction, 
the  mental  synthesis  of  elementary  cause-effect 
relationships,  logical  thinking  of  a  routine  nature, 
and  the  integration  of  test  results  with  pretest  hy¬ 
potheses.  Simple  sequential  tests  do  not  involve 
electro-cognition.  For  example,  continuity  tests 
would  not  involve  this  category.  This  is  a  "how 
to  make  it  work"  factor  as  opposed  to  electronic 
circuit  analysis,  which  is  a  "why  it  doesn't  work" 
factor.  Tasks  which  might  involve  electro-cog¬ 
nition  are: 

1.  making  logic  changes  in  a  data  processing  unit 

2.  comparing  an  output  waveform  to  a  manualized 
form 

3.  observing  fault  lights  and  inferring  module  to 
be  replaced 

This  factor  includes  the  motor  and  manipulative 
aspects  of  physically  repairing  a  component  which 
has  failed.  It  does  not  include  module  or  compo¬ 
nent  replacement  but  does  include  module  or  com¬ 
ponent  repair.  Examples  of  tasks  including  electro¬ 
repair  are; 

1.  replacing  a  broken  solder  joint 

2.  adjusting  the  contacts  on  a  relay 

This  factor  involves  teaching  others  how  to  inspect, 
repair,  operate,  or  maintain  electronic  equipments. 
Examples  of  tasks  involving  this  factor  are: 

1.  instructing  another  technician  on  how  to  use 
test  equipment 

2.  instructing  a  subordinate  on  how  to  perform  a 
test  or  repair  act 

3.  instructing  an  operator  on  how  to  work  an 
equipment 
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Table  1  (cont. ) 


4.  Electro- safety 


5.  Personnel 
Relationships 


6.  Electronic  Circuit 
Analysis 


This  factor  involves  implementation  of  special 
safety  procedures  so  as  to  minimize  personnel 
hazard  during  a  repair  or  so  as  to  minimize  the 
possibility  of  additional  equipment  damage.  An 
example  task  involving  this  factor  is; 

1.  observing  high  voltage  protection  instruc¬ 
tions  on  equipment 

This  factor  includes  the  management  and  super¬ 
visory  aspects  of  maintenance  organization  func¬ 
tions.  Examples  of  tasks  involving  this  factor  are; 

1.  supervising  the  operation/ inspection/ main¬ 
tenance  of  an  electronic  equipment 

2.  assigning  personnel  to  an  electronic  repair 

3.  developing  a  repair  schedule 

This  factor  is  purely  mental  in  nature.  It  includes 
the  application  of  electronic  principles  to  the  cor¬ 
rection  of  a  fault.  Electronic  principles  include 
the  selection  and  use  of  circuit  formulae  and  the 
application  of  the  results  of  calculations,  the  ap¬ 
plication  of  principles  of  electrical/  electronic 
functions,  and  the  like.  This  factor  is  different 
from  the  electro-cognition  factor  in  that  electro¬ 
cognition  is  almost  directly  effect-cause  related, 
whereas  electronic  circuit  analysis  involves  more 
sophisticated  consideration  of  intervening  processes. 
For  example,  if  in  the  case  of  a  faulty  output,  the 
technician  can  decide  that  either  module  A,  B,  or  C 
is  malfunctioning  and  that  he  can  complete  the  re¬ 
pair  by  sequential  replacement  of  modules  until  the 
correct  output  is  obtained,  then  the  factor  involved 
is  electro-cognition.  On  the  other  hand,  if  the  tech¬ 
nician  must  perform  a  test  on  each  of  A,  B,  and  C 
and  then  apply  Ohm's  Law  to  determine  the  faulty 
module,  then  electronic  circuit  analysis  is  involved. 
Electronic  circuit  analysis  might  also  be  involved 
in  certain  aspects  of  failure  reporting.  Examples 
of  electronic  circuit  analysis  are: 
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Table  1  (cont. ) 


7.  Equipment 
Operation 


8.  Using  Reference 
Materials 


9.  Equipment 
Inspection 


1.  determining  why  an  oscillator  yields  an 
improper  frequency  response 

2.  determining  why  the  time  delay  of  a  tim¬ 
ing  circuit  is  too  long 

3.  determining  why  a  power  supply  goes  into 
an  overvoltage  shutdown  mode 

This  factor  involves  the  operation  or  exercise  of 
prime  equipment  and  electrical  and  electronics 
test  equipment.  Examples  of  this  factor  are; 

1.  employing  repaired  equipment 

2.  using  an  oscilloscope 

This  factor  includes  the  use  of  supporting  docu¬ 
mentation.  The  use  of  schematics  and  block  dia¬ 
grams  is  included  under  either  electro-cognition 
or  electronics, circuit  analysis. 

This  factor  includes  inspections  of  electronic 
equipment,  including  those  inspections  and  ex¬ 
aminations  required  after  performing  a  correc¬ 
tion  or  repair  to  the  equipment. 


Reliability  of  Maintenance  Technicians  on  Each  Factor 

The  factorial  based,  empirically  derived  taxonomy  described 
above  provides  a  basis  for  describing  electronic  maintenance  tasks.  It 
provided  a  structure  which  is  manageable  and  relatively  unencumbered. 
As  such,  the  technique  was  used  in  studies  in  which  the  ability  of  Naval 
technicians  to  perform  the  functions  subsumed  by  the  factors  was  ex¬ 
amined.  In  order  to  obtain  evaluations  of  the  Naval  technician's  ability 
on  each  factor,  a  scheme  was  developed  which  was  drawn  from  the 
work  of  Whitlock  H963).  Whitlock  investigated  the  relationship  between 
observation  and  pei  lormance  evaluation  and  pointed  out  that:  (1)  perform¬ 
ance  evaluation  represents  a  response  to  observations  of  performance, 

(2)  observations  associated  with  performance  evaluation  are  observations 
of  performance  specimens,  and  (3)  observations  of  performance  speci- 
ments  can  be  rememhered  over  reasonable  rating  periods  and  reported 
accurately  at  the  end  of  the  rating  period.  A  reasonable  rating  period 
for  Whitlock  was  up  to  six  months  in  duration. 

Whitlock  defined  a  performance  specimen  as  "an  incident  of  rele¬ 
vant  performance  which  is  uncommonly  effective  or  uncommonly  ineffec¬ 
tive  [1963,  p.  15].  "  Regarding  the  definition  of  uncommon  performances, 
Siegel  and  Pfeiffer  (1966b)  pointed  out  that  this  definition,  in  a  sense, 
represents  an  adaptation  of  Flanagan's  critical  incident  technique  (cf. , 
Flanagan,  J.  C.  The  critical  incident  technique.  Psychological  Bulletin, 
1954,  51^,  327-358. )  Recall  of  such  incidents  will  depend  on  the  time  in¬ 
terval  between  the  event  and  the  rating  and,  in  part,  on  who  does  the  re¬ 
porting. 

Siegel  and  Pfeiffer  examined,  in  the  Naval  situation,  the  relation¬ 
ship  between  peer  ranking  of  personnel  proficiency  and  peer  estimates  of 
job  proficiency.  Peer  estimates  of  job  proficiency  were  determined, 
through  the  magnitude  estimate  method,  of  the  number  of  uncommonly  ef¬ 
fective  and  uncommonly  ineffective  performances  over  the  immediately 
preceding  four  month  period.  The  correlation  coefficient  obtained  be¬ 
tween  these  two  sets  of  data  was  .73.  The  researchers  commented 
that  since  peer  rankings  are  consistently  found  to  be  one  of  the  most  use¬ 
ful  indicators  of  personnel  proficiency,  the  correlation  of  .73  suggests 
that  the  judgments  of  uncommonly  effective  and  uncommonly  ineffective 
performances  possess  merit  for  the  same  purposes. 
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Siegel  and  Pfeiffer  (1966b)  employed  the  above  described  fac¬ 
tor  analytically  determined  job  activities  and  obtaiued,  for  each  ac¬ 
tivity,  magnitude  estimates  of  the  number  of  uncommonly  effective 
and  uncommonly  ineffective  performances  relative  to  a  short  prior 
period  for  avionic  personnel.  Two  Fleet  electronic  maintenance  ob¬ 
jectives  were  involved:  "readiness"*  and  "performance."**  The 
technicians  were  both  ratees  and  raters.  Each  technician  rated  the 
other  nine  in  the  f,roup.  Three  Naval  rates  were  involved:  aviation 
electronics  technician,  aviation  fire  control  man,  and  aviation  elec¬ 
trician's  mate.  A  ratio  of  uncommonly  effective  (UE)  performances 
divided  by  the  sum  of  uncommonly  effective  plus  uncommonly  ineffec¬ 
tive  (UI)  performances  (SUE/ SUE  +  EUI)  was  used  as  the  performance 
index.  This  index  yields  a  value  which  varies  between  0.  00  and  1.  00. 
The  researchers,  in  their  discussion  of  this  index,  claimed  that  it 
normalizes  across  the  opportunity  to  perform.  The  resultant  data 
tended  to  support  the  use  of  the  ratio,  in  that  emergent  differences 
were  in  the  anticipated  direction.  Specifically,  in  the  case  of  the  two 
squadrons  involved,  one  of  the  two  squadrons  possessed  technicians 
who  were  considerably  more  experienced  than  the  other.  The  ability 
of  the  technique  to  differentiate  in  this  manner  was  considered  as  evi¬ 
dence  supporting  its  discriminating  power.  Additional  significant  dif¬ 
ferences  were  indicated  between  technicians  and  between  job  activities. 
Although  there  was  no  a  priori  basis  for  predicting  the  direction  of 
these  differences,  Siegel  and  Pfeiffer  argued  that  these  results  also 
supported  a  contention  in  favor  of  the  discriminating  power  of  the  tech¬ 
nique.  Siegel  and  Pfeiffer  concluded  that:  (1)  magnitude  estimates  of 
uncommonly  effective  and  ineffective  performances  yielded  useful  data 
which  could  form  the  basis  for  a  personnel  subsystem  reliability  index, 
(2)  the  ratio  of  the  sum  of  uncommonly  effective  performances  to  the 
sum  of  the  uncommonly  effective  plus  uncommonly  ineffective  perform¬ 
ances  yields  an  index  which  distinguishes  in  the  anticipated  direction, 
and  (3)  the  obtained  avionic  personnel  subsystem  index  could  be  util¬ 
ized  for  posttraining  performance  appraisal,  personnel  placement,  and 
squadron  evaluative  purposes. 


♦Readiness:  Completion  of  any  given  mission  in  minimum 

time  with  an  appropriate  level  of  accuracy  and 
x'eliability. 

♦♦Performance;  To  maintain  self,  subordinate  personnel,  equip¬ 
ment,  and  systems  in  a  state  of  readiness  con¬ 
sistent  with  Fleet  requirements. 
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Siegel  and  Pfeiffer  (1966a)  also  provided  basic  information  re¬ 
garding  the  nature  and  characteristics  of  the  underlying  scale  when 
magnitude  estimates  are  employed  to  collect  performance  related  data. 
The  investigators  concluded  that  the  method  yields  a  scale  in  which 
standard  deviations  do  not  increase  with  mean  values  (metathetic  scale). 
These  data  again  suggest  the  utility  of  the  magnitude  estimate  method 
for  collecting  data  relative  to  job  performance. 

Siegel  and  Federman  (1970)  employed  the  same  factor  analyti¬ 
cally  derived  maintenance  job  dimensions  and  the  same  approach  (mag¬ 
nitude  estimates  of  uncommonly  effective  and  uncommonly  ineffective 
performances)to  obtain  data  on  a  Fleet  sample  of  533  technicians.  The 
technicians  represented  the  following  ratings:  electrician's  mate  (EM), 
electronics  technician  (ET),  fire  control  technician  (FT),  interior  com¬ 
munications  electrician  (IC),  radarman  (RD),  radioman  (RM),  sonar 
technician  (ST),  and  torpedoman's  mate  (TM).  The  data  were  based  on 
the  following  destroyers:  USS  Roan,  USS  Dyess,  USS  Sperry,  USS 
Basilone,  USS  Ingraham,  USS  Page,  USS  Fiske,  USS  Eaton,  USS  Cony, 
USS  Hank,  and  USS  Conway.  These  destroyers  were  based  at  Philadel¬ 
phia  and  Newport. 

The  data  obtained  on  these  technicians  were  treated  in  the  same 
manner  (SUE/  SUE  +  SUI)  as  in  the  Siegel  and  Pfeiffer  (1966b)  study. 
The  data  were  analyzed  separately  for  each  rate  by  pay  grade,  ship, 
and  squadron. 

Several  of  Siegel  and  Federman's  conclusions  were  that:  (1)  a 
statistically  significant  difference  existed  among  the  electronics  rat¬ 
ings,  (2)  no  significant  difference  was  found  among  the  three  squdrons, 
(3)  the  job  activities  differed  significantly  among  themselves,  and  (4) 
there  were  no  significant  differences  among  the  11  ships  sampled. 

Since  there  were  no  significant  differences  among  ships  and  squadrons, 
the  data  were  combined  acroos  ships  and  squadrons.  Figure  2  presents 
the  combined  data  for  the  individual  ratings  on  each  job  activity. 
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If  the  position  is  taken  that:  (1)  the  estimates  of  unusually  ef¬ 
fective  and  unusually  ineffective  performance  represent  the  tails  of  a 
distribution,  (2)  the  remaining  performances  (viz.,  that  are  neither 
unusually  effective  or  unusually  ineffective)  will  distribute  themselves 
equally  over  the  central  area  of  the  distribution,  and  (3)  performances 
above  the  mean  of  the  distribution  are  satisfactory,  whereas  perform¬ 
ance  below  the  mean  are  unsatisfactory,  then  the  numeric  yielded  by 
the  ratio  SUE/ SUE  +  SUI  can  be  held  to  represent  the  probability  of 
satisfactory  performance  by  a  member  of  a  given  Naval  rating  on  a 
given  job  factor.  It  is  this  set  of  probability  numbers  which  forms  the 
basic  data  for  the  present  study. 


Method  for  Compounding 


Several  different  job  activities  and  persons  may  be  involved 
in  the  performance  of  a  task.  Task  performance,  to  be  successful, 
may  require  the  successful  performance  of  all  involved  job  activities 
by  all  persons  or  the  successful  performance  of  some  particular  com¬ 
bination  of  job  activities.  The  probability  of  successful  performance 
will  be  increased  when  several  persons  are  assigned  to  the  same  task, 
i.  e. ,  when  they  perform  a  job  activity  in  parallel  with  each  other.  In 
addition,  the  probability  of  successful  performance  will  be  increased 
if  a  technician  is  permitted  to  repeat  the  performance  or  one  aspect  of 
the  performance. 

Siegel  and  Miehle  (1967)  presented  methods  for  determining  over¬ 
all  probability  of  successful  task  accomplishment  when  the  probability 
of  accomplishing  each  of  the  elements  of  the  task  is  known.  The  meth¬ 
ods  suggested  are  reviewed  below.  The  methods  were  developed  to  ex¬ 
ploit  the  job  factors  isolated  in  the  multidimensional  scaling  studies, 
as  well  as  the  personnel  reliability  data  on  each  factor  as  collected  origi¬ 
nally  by  Siegel  and  Pfeiffer  and  later  by  Siegel  and  Federman. 


Let: 


s  =  satisfactory  task  performance 


r  =  satisfactory  performance  of  job  activity  m 

by  technician  n 


Rmn  =  reliability  of  technician  n  on  activity  m 

P  [r  1  =  probability  that  statement  r  is  true. 
rl  mn  r  mn 


Thus  P„[rmn]  =  ^mn  an(*  Prts]  =  reliability  of  task  performance. 

Suppose  performance  of  a  task  involves  technician  b  on  three  job 
factors:  3,  4,  and  6  and  technician  g  on  three  factors:  3,  5,  and  8.  Both 
technicians  perform  factor  3.  The  condition  for  satisfactory  task  per¬ 
formance  is: 


s 

V 

A 


=  <r3b  V  r3g>  A  r4b  A  r6b  A  r5g  A  r8g 

is  a  symbol  for  inclusive  or  (inclusive  disjunction) 

is  a  symbol  for  and  (conjunction) 

is  a  symbol  for  "is  equivalent  to.  " 


We  are  not  limited  to  an  "and"  and  "or"  logic.  Statements  could  con¬ 
ceivably  be  connected  by  conditional  or  biconditional  symbols.  These  in 
turn  can  be  expressed  in  items  of  "and,  "  "or,  "  and  negation.  The  nega¬ 
tion  of  is  ry 

If  all  activities  must  be  performed  satisfactorily,  the  condition 
is  expressed  by  joining  all  statements  by  "conjunc'ion"  (A),  s^  r^a  A  r  . 
This  might  be  called  a  series  task. 


Prts)  '  Pr[rla  A  r2J  =  Pr[rlalr2a!  Pr[r2a! 
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Pr[rla|  r2a]  is  a  conditional  probability  which  is  read  "the  probability  of 
rla»  given  r2a»  "  K  is  the  probability  that  rja  is  true  under  the  condition 
of  r2a  being  true.  When  the  truth  of  rja  is  independent  of  the  truth  of  r2a 
we  say  that  rja  and  r2a  are  independent  statements.  In  this  case,  Pr[s] 

PrtrlJPrtr2aJ- 


Let  5  =*r2aAr3dAr6a 


PrW  *  Pr[r2aAr3dAr6d> 


'  Pr^r2a  I  r3d  A  r6a'  Pr^r3d  I  r6a'  Pr^  r6a^ 

If  all  statements  are  independent,  this  reduces  to: 

P  [s]  =  P  [r.  ]P  [r  ]  P  [r.  ]. 
rL  1  rL  2aJ  rl  3dJ  r  L  6aJ 

If  the  satisfactory  performance  of  one  or  the  other  factor  (or  both) 
is  required  for  the  satisfactory  performance  of  the  task,  then  the  task  is 
called  a  parallel  task.  The  satisfactory  performance  of  parallel  tasks  is 
expressed  as; 


s  *  r3e Vr7e’ 

In  this  case,  job  factors  3  and  7  are  involved  and  the  task  is  performed 
by  technician  e. 

When  the  same  job  factor  is  performed  by  two  technicians  and  ac¬ 
ceptable  performance  of  either  technician  will  constitute  acceptable  per¬ 
formance  for  the  team,  the  condition  is  expressed  as; 


s 


r3aVr3c‘ 


This  condition  is  also  referred  to  as  a  parallel  performance.  Here,  job 
factor  3  is  performed  by  technicians  a  and  c. 
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Pr[s)  *  Pr(r3a  V  r3c]  m  Pr«r3a  A  r3c)'1  =  1  '  prIr3a  A  ry 

=  1'Prtr3alr3clPrtr3cl 

■'  1  '  (1  '  Pr'r3al  r3c1,(1  ’  Pr[r3c) 

If  the  statements  are  independent,  then: 

Pr[s]  *  1  -  (1  -  Pr[r3a])U  -  Pr[r3c]). 

Let  s  «  rlb  v  v 

pr[SJ  ■  Pr(rlb  V  rlc  V  rlgJ  .  ,  -  a  rJc  A  ry] 

=  1-Pr[riblrlcArlgJPr[rlc|rlg]Pr(l.ibj 

'  1  "  (1  ’  P--[rlb|rlc  A  ry)(1  -  Prfrlclriy,(1  •  Pr[rig]- 


For  independent  statements,  this  reduces  to- 


Pr[s]  *  1  -  (1  -  Pr[rib])(l  -  PrtrIc])(l  -  Pr[rlg]). 

outers  “ofa^uefo ?"  * 

many  differed!  forras  if  conditional  pVobabmtieaT^TnX^.  Wr“ten  *“ 
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Example  1.  Assume  that  Task  A  is  performed  by  technician  c 
and  that  job  factors  1,  3,  4,  7,  and  9  are  involved.  Performance  will 
be  considered  satisfactory  if,  and  only  if,  either  (or  both)  factors  1  or  7 
are  performed  satisfactorily,  either  (or  both  factors  3  and  9  are  per¬ 
formed  satisfactorily,  and  factor  4  is  performed  satisfactorily.  This 
is  symbolized  by: 


s  ~  (r  V  r  )  A  (r  V  r_  )  A  r  * 
lc  7c  3c  9c  4c 


.  P  ,«rlc  V  rTc)  | <r3o  V  ,  9c>  A  r  ]  P^(r  V  r^)  |  r  ]  P,i< 


The  first  two  probability  expressions  would  require  further  ex¬ 
pansion  to  remove  the  expression  r ^  V  r~  in  the  first  term  and  the  ex¬ 
pression  rg  .  V  rgc  in  the  second  term.  Tnis  would  produce  a  very  com¬ 
plicated  appearing  expression,  still  containing  conditional  probabilities, 
for  P  [s].  However,  it  can  be  argued  that  such  expansion  is  not  war¬ 
ranted. 


In  the  study  of  systems  reliability,  it  is  generally  assumed  that 
the  proper  operation  of  one  component  does  not  depend  on  the  proper  oper¬ 
ation  of  another.  This  assumption  does  no1  always  hold.  For  example, 
suppose  that  two  beams  are  used  to  support  a  weight.  If  one  beam  fails, 
the  whole  weight  is  then  placed  on  the  other  which  will  now  possess  n 
greater  probability  of  failure,  although  each  beam  was  designed  to  bold 
the  whole  weight.  This  is  the  "domino  effect.  " 


*This  expression  might  be  read  as  follows:  ihis  maintenance  task 
will  be  successfully  performed  if  either  electro-cognition  or  equipment 
operation  is  completed  successfully,  and  either  an  instruction  or  an 
equipment  inspection  is  performed  acceptably  and  safety  precautions 
arc  observed  throughout.  Vll  activities  are  performed  by  technician  r. 


19 


For  independence  to  hold,  a  failure  of  one  component  must  not 
influence  the  operation  of  another.  If  there  is  a  cause  for  failure  of  one 
component,  that  cause  should  not  operate  on  the  other  components.  If, 
for  satisfactory  overall  performance,  all  components  must  operate 
properly,  then  when  one  fails,  the  whole  system  fails.  In  this  case,  it 
is  irrelevant  whether  other  components  also  fail  as  a  result  of  the  fail¬ 
ure  of  the  first  component.  Here,  the  reliability  value  of  interest  is 
the  conditional  probability  of  proper  functioning,  given  that  all  other 
components  function  properly.  Usually  a  component  is  tested  in  isola¬ 
tion,  and  it  is  assumed  that  when  combined  with  other  components  its 
reliability  will  not  be  influenced.  Otherwise,  each  component  would 
have  to  be  assigned  as  many  reliability  values  as  there  are  systems  in 
which  it  is  used. 

This  consideration  may  also  hold  for  persons  on  a  job  factor. 

If  reliabilities  were  not  independent,  then  a  single  value  such  as  r 
would  be  useful  only  if  technician  b  worked  on  job  factor  all  by  himself. 
If  technician  b  performed  several  job  activities  or  worked  with  other 
technicians,  then  his  reliability  would  be  determined  on  a  particular 
task  under  a  variety  of  "given  conditions.  "  Overall  task  reliability 
would  then  be  determined  directly  rather  than  on  the  basis  of  the  com¬ 
ponent  reliabilities. 

Assuming  independence,  i.  e. ,  that  success  or  failure  of  one  job 
activity  does  not  affect  the  probability  of  success  or  failure  of  another 
job  factor,  the  formula  simplifies  to: 


pM  ■  Pr[r,c  A  r?c]  P^,  A  rgc]Pr[r4c] 

PrN=  {1  -  d-Pr[rlc])(l  -  Pr[r7c])}{l  -  (l-Pr[r3c])(l-Pr[r9c])}Pr[r4c). 


If  the  reliabilities  (SUE/ SUE  +  SUI  values)  for  technician  c  on  job 
factors  1,  3,  4,  7,  and  9  are  .  88,  .  82,  .  88,  .  82,  and  .  89,  respectively, 
then  the  overall  probability  of  successful  task  performance  is: 
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Pr[s]  =  (1  -  (1  -  .  88)(1  -  .  82)}  {1  -  (1  -  .82)(1  -  .  89)).  88 
=  (l  -  (.  12)(.  18)}{l  -  (.  18)(.  11)}.  88 
=  (l  -  .  02 16 }{l  -  .  0198)}.  88 
=  .  845. 

Example  2.  Assume  that  Task  B  involves  technician  j  who  per¬ 
forms  job  factors  1,  2,  5 ,  7 ,  and  technician  e,  who  performs  job  factor  6. 
Both  technicians  work  on  job  activities  3,  4,  and  8. 


s—  r  .  A  r  .  A  r  Ar_ .  A r  A  (r  V  r  )A  (r  V  r  )  a  (rQ  .  V  r  ). 
i  J  5j  7j  6e  3j  3e  4j  4e  8j  8e 


P  [s]  =  P  [rlP  [r  IP  [rlP  [r  ]  P  [r  ] 
r  rL  lj  rl  2j  rl  5j  rl  7j  rL  6e 


(i  -  (1  -  Pr[r3.])(l  -  Pr[r3e])}  (1  -  (1  -  Pr[r4j])(l  -  Pr[r4e])} 

{1-(1-Pr[r8.])(l-P.[r8e]} 

Pr[s]  =  (.  82)(.79)(.91)(.  77)(.  86)  (l  -  (1  -  .  81)(  1  -  .  76)} 

{1  -(1-.87K1  -.80)}  (1  -(1  -.88)(1  -  .75)} 

=  (,82)(.79)(.91)(.77)(.86)  (l  -(.  19)(.24)} 

(1  -  (.  13)(.  2)  1  -  (.  12)(.  25)} 

=  (.  82)(,  79)(.  9 1 )( .  77)(.  86)(  1  -  .  0456)(1  -  .026)(1  -  .  03) 

=  (.  82)(.  7 9 ) ( .  91)(.  77)(.  86){.  9544H.  97  4)(.  97) 


=  .  352. 
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Job  Activity  Repetition 


It  is  often  possible  to  improve  the  unsatisfactory  perform¬ 
ance  on  a  job  by  repeating  a  process  or  by  calling  on  another  tech- 
niciar,  However,  in  order  for  reliability  to  be  affected  by  the 
repetition  of  a  job  activity,  the  realization  of  the  unsatisfactory 
performance  must  come  prior  to  the  completion  of  the  entire  task. 
Otherwise,  the  entire  job  performance  may  be  considered  unsatis¬ 
factory.  The  repetition  of  job  activities  is  equivalent  to  parallel 
operations  which  yield  reliability  factors  of:  1  -  (1  -  R)(l  -  R)  = 

1  -  (1  -  2R  +  R?  )  *  2R-R2  =  R(2  -  R),  instead  of  R  itself.  Thus,  if 
R  =  .  8,  the  new  reliability  factor  is:  .  8(2  -  .  8)  =  .  8(1.  2)  =  .  96. 

The  expected  number  of  attempts,  E,  is  a  function  of  the  max¬ 
imum  number  (n)  of  attempts  permissible  or  the  number  of  attempts 
necessary  to  give  a  specified  resultant  reliability. 

P  -  *  -U-R>n 


where  n  is  the  maximum  permissible  number  of  trials. 

In  the  limiting  case,  as  n  increases  indefinitely,  E  approaches 
1/R.  Thus,  if  R  =  .  8,  E  approaches  1.25.  This  means  that,  if  many 
trials  are  allowed,  or  equivalently,  if  the  required  reliability  must  be 
close  to  1,  then  for  R  =  .  8  the  average  increase  in  number  of  trials  is 
not  more  than  25  per  cent.  For  R  =  .6,  E  approaches  1.67. 

Let  Rn  equal  the  reliability  attained  by  allowing  up  to  n  trials: 

R  =R  +  R(1-R)  +  R(1-  R)2  +  . .  .  +  R(1  -  R)0"1 
n 

=  1  -  (1  -R)n. 

This  formula  also  applies  when  n  technicians  work  in  parallel  on  the 
same  job  activity. 
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Computer  Simulation 

The  computational  technique  described  above  in  association 
with  the  Fleet  data  of  the  Siegel  and  Federman  (1970)  study  provides 
a  method  for  estimating  the  probability  that  a  maintenance  technician 
will  satisfactorily  perform  a  given  task.  The  question  of  how  long  it 
will  take  the  maintenance  technician  to  complete  the  repair  task  is 
now  considered.  For  this  purpose,  the  present  program  relies  on  a 
computer  simulation  technique.  The  specific  computer  simulation  in¬ 
volved  is  the  model  known  as  the  Siegel -Wolf  man-machine  simulation 
model  (Siegel  &  Wolf,  1969). 


Computer  simulation  has  had  a  wide  and  varied  background  of 
utilization.  It  has  been  used  to  investigate  investment  behavior  in  the 
stock  market,  plant  flow,  and  social  behavior.  It  has  also  been  used 
to  test  a  number  of  military  systems,  from  the  man-machine  interac¬ 
tive  point  of  view.  Digital  computer  simulation,  made  possible  by  the 
advent  of  the  high-speed  digital  "nmpiTi  ~rp  possesses  at  least  the  follow 
ing  attributes  and  advantages: 


•  allows  consideration  of  the  idiosyncratic  and 
variable  aspects  of  human  performance 

•  often  costs  less  than  physical  simulation 

•  allows  hypothetical  procedures  and  systems 
to  be  tested 

•  facilitates  the  consideration  of  myriad  vari¬ 
ables  in  interaction 
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i 

:9 

Over  the  years,  the  Siegel -Wolf  model*  has  been  successively 
revised,  improved,  and  validated.  Initially  prepared  to  simulate  the 
actions  of  a  single  operator,  the  model  was  later  expanded  to  allow 
simulation  of  a  two-operator  man-machtne  system.  New  variables 
were  added  and  computational  routines  modified  and  updated  as  the 
model  matured.  Similarly,  the  model  has  been  continuously  updated 
in  terms  of  more  modern  computing  equipment  and  in  terms  of  more 
symbolic  programming  languages.  Prepared  originally  for  the  IBM  650 
computer  system  using  the  SOAP  programming  technique,  the  model 
was  later  programmed  in  AUTOCODER,  and  more  recently,  it  was  pre¬ 
pared  in  FORTRAN  IV. 

The  Siegel -Wolf  model  has  been  documented  in  professional 
journals,  addresses  to  professional  associations,  and  technical  reports. 

The  most  up-to-date  description  of  the  technique  is  found  in  Siegel  and 
Wolf  (1969).  Other  documentation  is  summarized  in  Tables  2  and  3.  Ad¬ 
ditional  documentation,  expansion,  and  elaboration  of  the  technique  has 
been  produced  by  the  Boeing  Company,  Honeywell,  the  General  Electric 
Company,  Autonetics,  the  Naval  Air  Development  Center,  and  the  Aero¬ 
space  Medical  Research  Laboratory  of  the  Wright-Patterson  Air  Force 
Base. 


*The  word  "model”  as  used  here  is  defined  as  a  logical  math¬ 
ematical  representation  of  a  concept,  system,  or  operation  program¬ 
med  for  solution  on  a  high-speed  digital  computer  (Martin,  1968).  As 
such,  digital  simulation  models  are  distinct  from  replication  (analogy) 
and  formulation  (mathematical)  models. 
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Summary  of  Reports  on  Man-Machine  Simulation 


Table  3 

ry  of  Addresses  on  Man-Machine  Simulation 
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The  validity  of  the  model's  predictions  has  been  demonstrated 
in  a  wide  range  of  applications.  The  validational  studies  completed  at 
Applied  Psychological  Services  included  carrier  landing,  air-to-air 
missile  firing,  in-flight  refueling,  air  intercept,  sonar  employment, 
and  a  series  of  other  simulated  man-machine  interactive  situations. 
Additional  validity  studies  were  completed  by  the  Boeing  Company 
(Outcalt  et  al. ,  1966)  and  Honeywell  Incorporated  (Lane  et  al.  ,  1966). 
The  ability  of  the  model  to  predict  independent  outside  criteria  data 
was  tested  in  all  of  these  valicrational  studies.  With  one  exception, 
the  results  of  all  validational  efforts  reveal  satisfactory  correspond¬ 
ence  (differences  which  are  not  statistically  significant)  between  the 
model's  prediction  and  the  criterion  data.  Additionally,  certain  of  the 
model's  internal  constructs  were  validated,  and  the  ability  of  the  mod¬ 
el  to  predict  part-task  success  was  verified.  The  model  has  been 
adapted  and  successfully  employed  by  a  number  of  industrial  and  gov¬ 
ernmental  organizations.  Thus,  the  model  has  withstood  reasonable 
tests  of  both  validity  and  utility.  Complete  descriptions  of  *hese  prior 
tests  and  applications  are  found  in  Siegel  and  Wolf  (1969)  and  in  the 
sour  ces  listed  in  Table  2. 


Overview  of  the  Moc  el 


The  model  provides  the  capability  to  simulate  stochastically 
the  acts  and  behaviors  of  a  human  as  he  performs  the  sequence  of 
subtasks  associated  with  the  performance  of  a  total  task.  On  the  basis 
of  task  analytic  input  data,  the  model  sequentially  mimics  the  "per¬ 
formance"  of  each  subtask  by  each  per  son  simulated.  The  nor  mal  se¬ 
quence  of  subtasks,  whether  linear  or  non-linear,  may  be  modified  if 
actions  have  to  be  skipped  or  repeated  due  to  failure  of  a  subtask  by 
the  operator  or  as  a  r  esult  of  operator  decisions.  During  the  cour  se 
of  the  computer's  "performance"  of  the  task,  results  ar  e  recorded 
indicating  the  area  of  operator  overload,  failur  e,  idle  time,  peak 
stress,  etc.,  for-  the  given  set  of  selected  parameter  s.  The  model's 
output  of  principle  concern  in  the  present  context  is  the  time  for-  task 
completion.  The  model  records  the  amount  of  time  involved  in  com¬ 
pleting  each  subtask  in  the  total  task,  the  cumulative  time  for-  each 
subtask  in  the  total  task,  and  the  total  time  for  total  task  completion. 
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The  major  feature  of  the  model  of  interest  in  the  current  context 
is  calculation  of  task  execution  time.  The  following  important  items  are 
also  calculated  or  otherwise  considered  in  the  model; 

•  task  precedence  (variable  sequencing  of  tasks) 

•  maximum  stress  encountered 

•  operator  interaction  (waiting  for  a  partner) 

•  joint  tasks  (performed  by  both  operators  simul¬ 
taneously) 

•  equipment  delays 

•  operator  decisions 

•  skipping  of  nonessential  tasks 

•  operator  cohesiveness 

•  idle  time  spent  waiting  for  a  prespecified  event 

•  time  allotted  for  the  mission 

•  time  precedence  (idling  until  a  given  time  occurs) 

The  simulation  of  intra  and  inter  individual  differences  in  per¬ 
forming  any  individual  task  is  based,  in  part,  on  a  random  process. 

The  total  simulation  process  is  a  repetitive  process,  i.  e. ,  tasks  are 
simulated  sequentially  to  comprise  a  task  trial  and  the  task  trials  are 
repeated  (iterated)  many  times  to  obtain  averages  of  the  data  generated 
by  randomization  techniques. 

The  nature  of  the  model  is  such  that  specific  subtask  input  data, 
independent  of  parameter  data,  are  required.  Each  discrete  subtask  is 
described  for  each  technician  by  tne  data  given  in  Table  4.  A  limit  of 
300  'jubtasks  for  each  technician,  prepared  one  per  caid  in  accordance 
with  the  format  of  Table  4,  has  been  established. 
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The  following  paragraphs  describe  and  explain  some  of  the  data 
specified  in  Table  4.  A  joint  task  (type  J)  is  one  performed  simultane¬ 
ously  by  both  technicians.  An  equipment  task  (type  E)  is  introduced  to 
produce  a  delay  in  task  performance  due  to  a  factor  other  than  human 
performance.  No  stress  functions  are  calculated  for  the  type  E  tasks. 

A  decision  task  (type  D)  is  incorporated  into  the  sequence  to  allow 
branching,  skipping,  or  looping  in  the  performance  sequence.  If  the 
current  subtask  is  a  success,  subtask  (i,  j)s  is  selected  with  a  prob¬ 
ability  of  py.  If  the  current  subtask  fails,  the  next  subtask  selected 
is  subtasl:  (i,  j)f  with  a  probability  1-Pij  •  A  "cyclic"  subtask  (type  C) 
provides  for  the  case  in  which  a  subtass.  cannot  be  started  until  the 
next  cycle  time  in  a  series  of  equitemporal  intervals. 

The  subtask  precedence  input,  dy,  (mnemonic  delay)  repre¬ 
sents  a  subtask  number  which  must  be  successfully  completed  by  the 
partner  before  a  technician,  j,  can  begin  his  own  subtask  i.  The  time 
precedence,  (Iy),  is  the  time  from  mission  start  before  which  a  tech¬ 
nician  is  not  permitted  to  begin  a  subtask.  The  values  of  essential  and 
nonessertial  time  remaining,  and  T^  are  calculated  after  other 
performance  data  are  available,  according  to  the  procedure  given  in 
Siegel  and  Wolf  (1969). 

Two  Special  Jump  Subtasks  are  provided.  Special  Jump  Sub¬ 
task  1  enables  both  technicians  to  jump  to  an  individually  specified  sub¬ 
task  if  one  of  the  technicians  ignores  the  Special  Jump  Subtask  type  1. 

If  a  subtask,  so  identified,  is  ignored  (due  to  stress  levels),  operator  j 
will  go  to  NXTJ(I,  J)  for  his  next  subtask,  and  his  partner  (j')  will  go  to 
NXTJP(I,  J).  If  the  subtask  is  not  ignored,  NXTS(I,  J)  and  NXTF(I,  J) 
apply  as  usual.  Special  Jump  Subtask  type  2  provides  a  team  decision 
capability  to  the  model.  If  a  subtask,  so  identified,  is  a  success  (prob¬ 
ability  =  p^),  then  technician  i  will  go  to  NXTS(I,  J)  for  his  next  subtask. 
However,  if  the  subtask  is  failed,  tecnnician  j  goes  to  NXTJ(I,  J)  and 
his  partner  goes  to  NXTJP(I,  J).  In  these  ways,  one  technician  can  make 
a  decision  which  will  determine  tile  future  sequence  of  subtasks  for  both 
technicians. 
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Four  parameters  are  provided  to  the  model  on  input  cards  in 
the  format  shown  in  Table  5.  A  pair  of  cards,  one  card  for  each  oper¬ 
ator,  is  supplied  for  each  computer  run  desired.  A  run  is  composed 
of  the  simulation  of  N  task  performances  (iterations).  The  stress 
threshold  is  considered  the  operator's  "breaking  point.  "  The  model's 
simulation  is  based  on  the  psychological  concept  that  time-pressure 
stress  organizes  behavior  up  to  a  threshold  point  and  disorganizes  it 
beyond  that  point.  Stress  is  calculated  as  the  ratio  of  the  average  time 
to  complete  the  remaining  essential  subtasks  to  the  total  time  remain¬ 
ing  available  to  the  operator.  A  value  for  stress  greater  than  unity  will 
increase  both  his  speed  and  success  probability.  At  the  threshold,  the 
effect  of  stress  is  reversed,  simulating  disorganization  and  confusion 
of  the  operator. 

The  F  factor  for  each  operator  is  an  individuality  indicator  repre¬ 
senting  operator  speed  or  proficiency.  An  average  operator  is  given  an 
F  factor  of  unity;  faster  and  more  proficient  operators  have  lower  values. 

The  third  parameter  pair  is  the  toted  time  available  to  complete 
task  performance --a  time  limit.  Both  operators  must  finish  before 
their  time  limit  is  reached  in  order  that  an  iteration  can  be  considered 
successful. 

The  fourth  parameter  pair  is  the  period  of  time  applicable  to 
cyclic  (type  C)  subtasks.  A  cyclic  subtask  will  be  initiated  only  at  a 
time  which  is  a  multiple  of  the  period,  P.  If  necessary,  the  simulated 
operator  will  wait  until  such  a  time  occurs. 

The  task  and  parameter  input  data  are  preceded  in  the  computer 
input  deck  by  three  cards  which  supply  information  on  the  general  con¬ 
ditions  of  the  runs.  The  card  format  and  contents  of  these  cards  are 
shown  in  Table  6. 

Figure  3  displays  pictorially  the  sequence  of  cards  in  the  input 
deck  by  card  type.  A  maximum  of  300  subtask  input  data  cards  are  per¬ 
mitted  per  operator. 
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General  Run  Information  and  Card  Format 
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CARD 

NAME 


-BLANK 


PLOTTING  INFORMATION 

BLANK 

PARAMETER  INPUT  DATA 
(OPERATOR  CARDS) 


PARAMETER  INPUT  DATA 
(OPERATOR  CARDS) 


MISSION  INPUT  DATA 
(TASK  ELEMENT  CARDS  FOR 
BOTH  OPERATORS) 


NUMBER  OF  TASK  ELEMENT 
CARDS 

GENERAL  RUN  INFORMATION 


TITLE  INFORMATION 


A  variety  of  tabular  data  are  available  from  the  model.  These 
data  are  typically  recorded  on  magnetic  tape  during  the  runs  for  sub¬ 
sequent  printout  on  a  high-speed  line  printer.  The  following  categories 
of  results  are  available: 


Title 

Frequency 

Remarks 

detailed  listing 

every  task 

optional,  1  line  per  task 

Iteration  suanary 

end  of  each  Iteration 

optional,  7  lines  per  iteration 

run  suaaary 

end  of  run 

9  lines  plus  several  frequency 
distributions  (1  line  per  task) 

plot 

end  of  a  series  of  runs 

1  page  per  p lot 

In  the  optional  detailed  listing,  the  following  data  are  recorded 
for  each  operator: 

1.  subtask  number 

2.  type  of  subtask 

3.  essentiality  (indicator)(N  =  nonessential,  otherwise 
blank) 

4.  stress  as  well  as  augmented  (total)  stress  (aug¬ 
mented  stress  is  a  stress  value  for  one  operator 
based  on  his  stress  value  and  that  of  his  partner) 

5.  waiting  time,  time  spent  waiting  for  partner 

6.  subtask  execution  time 

7.  cumulative  subtask  execution  time 
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8.  result  indicator 


blank  =  success 
F  =  failure 
I  =  ignore 

9.  cohesiveness  indicator  -  a  factor  indicating 
team  cohesiveness  based  upon  the  stress  and 
stress  thresholds  of  each  operator 

The  following  information  is  optionally  displayed  in  the  itera¬ 
tion  summary  for  each  of  the  N  iterations  in  a  run: 

1.  iteration  number 

2.  run  number 

3.  trial  number(indicator  for  a  series  of  runs) 

4.  result  indicator 

overrun  =  task  failure  (time  overrun) 
underrun  =  task  success 

5.  total  time  used  -  the  larger  of  the  total  time  used 
by  either  operator 

6.  the  following  data  are  provided  for  each  operator; 

a.  operator  number 

b.  stress  threshold,  Mj 

c.  speed  factor,  Fj 

d.  time  available,  Tj 

e.  time  used  for  this  iteration 

f.  difference  (time  available  -  time  used) 

g.  total  waiting  time  (does  not  include 
item  1  below) 

h.  value  of  highest  (peak)  stress 

i.  the  task  number  on  which  the  peak  stress 
occurred 

j.  stress  at  end  of  iteration 

k.  cohesiveness  at  end  of  iteration 
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1. 


m. 


n. 


o. 


P. 

q. 


time  spent  in  waiting  for  the  period  on 
cyclic  tasks 

goal  orientation  -  difference  at  end  of 
iteration  (goal  -  performance) 
goal  orientation  -  maximum  difference 
in  goal  orientation  during  iteration 
goal  orientation  -  minimum  difference 
in  goal  orientation  during  iteration 
performance  at  end  of  mission  iteration 
initial  input  goal  orientation  value  (task  1) 


The  results  of  each  run  (set  of  iterations  with  constant  parameter 
values)  are  displayed  in  tabular  form.  It  contains  the  following: 

1.  run  number 

2.  total  number  of  iterations  performed,  N 

3.  number  of  successful  iterations 

4.  per  cent  successful  iterations 

5.  time  available,  T 

6.  the  following  data  are  listed  for  each  operator; 

a.  operator  number  i 

b.  stress  threshold,  Mj 

c.  speed  factor,  Fj 

d.  time  available,  Tj 

e.  average  time  used  over  N  iterations 

f.  average  difference  (time  available  - 
time  used) 

g.  average  waiting  time 

v  h.  average  peak  stress 

•  i.  average  final  stress 

j.  average  cyclic  waiting  time 
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7.  the  following  frequency  distributions  are  displayed 
and  values  presented  for  each  task.  The  items 
identified,  below,  with  an  asterisk  are  also  totaled 
and  averaged,  per  iteration,  over  all  tasks: 

a.  subtask  number 

b.  last  subtask  completed  before 
finishing  the  iteration  or  running 
out  of  time- -for  both  operators 

*c.  count  of  the  number  of  subtasks 
failed --for  each  operator 
*d.  count  of  the  number  of  subtasks 
ignored- -for  each  operator 
*e.  time  spent  in  performing  subtasks 
which  were  failed 

f.  count  of  the  number  of  subtasks  for 
which  the  peak  stress  occurred- -for 
both  operators 

g.  time  from  beginning  of  mission  that 
the  subtask  was  completed  on  the  aver¬ 
age — for  each  operator 

h.  average  stress  prior  to  beginning  each 
subtask- -for  each  operator 

i.  average  cohesiveness  value- -for  each 
operator 


In  order  to  employ  the  model  in  the  present  context,  a  number  of 
changes  were  implemented  in  the  form  of  an  independent  subroutine.  The 
changes  serve  to  adjust  the  input  data  in  such  a  manner  that  the  subtask 
success  probability  for  each  subtask,  in  a  total  maintenance  task,  is  con¬ 
cordant  with  the  factor  reliability  values  presented  in  Figure  2.  This  pro¬ 
gram  modification  was  implemented  by  means  of  an  independent  preproces¬ 
sor,  and  thus  the  structure  of  the  simulation  model  (and,  accordingly,  its 
validity),  per  se,  was  left  unmodified. 
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Specifically,  the  preprocessor  takes  the  factor  reliability  val¬ 
ues  and  allocates  them  across  the  sequence  of  subtasks  involved  in 
performing  a  maintenance  task.  The  specific  factor  reliability  values 
chosen  depend  on  the  Naval  rating  whose  performance  is  to  be  simu¬ 
lated  (i.  e. ,  if  a  technician  in  the  radioman's  rating  is  to  be  simulated, 
the  factor  reliability  values  for  this  rating  are  allocated).  The  per¬ 
formance  of  a  specific  repair  is  then  simulated  within  the  Siegel -Wolf 
model  to  yield  an  estimate  of  the  time  for  malfunction  correction.  The 
details  of  the  preprocessing  subroutine  are  presented  in  Chapter  II  of 
the  present  report. 

When  the  computer  simulation  and  the  computational  technique 
are  both  applied  to  the  same  malfunction  correction,  the  end  result  is 
a  statement  of  time  for  malfunction  correction  (from  the  stochastic  digi¬ 
tal  simulation)  and  of  the  probability  of  satisfactory  performance  (from 
the  computational  technique). 
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CHAPTER  U 


METHODS  AND  PROCEDURES 


Chapter  I  of  the  present  report  reviewed  the  logic  of  a  set  of 
complementary  methods  for  predicting  human  performance  reliabil¬ 
ity  in  regard  to  electronic  maintenance  situations.  Chapter  I  also  de¬ 
scribed  the  results  of  efforts  completed  toward  this  goal  prior  to  the 
present  program.  The  purposes  of  the  present  program  were  to  try 
these  methods  in  a  Navy  context  in  order  that  an  indication  might  be 
gained  of  the  general  applicability  and  reasonableness  of  the  approach¬ 
es  described  above.  To  this  end,  two  current  Navy  systems  were  se¬ 
lected,  the  Radio  Set  AN/URC-35  and  the  Radar  Set  AN/APS-115. 

These  two  electronic  equipments  are  described  in  Chapter  II.  In  addi¬ 
tion,  the  methods  are  described  which  were  employed  to  select  a  mal¬ 
function  correction  sample  which  would  provide  a  basis  for  evaluating 
the  reasonableness  and  appropriateness  of  the  proposed  deterministic 
procedure  for  predicting  the  probability  of  successful  malfunction  cor¬ 
rection  and  cf  the  stochastic,  digital  computer  procedure  for  predict¬ 
ing  the  time  to  perform  the  sequence  of  subtasks  associated  with  mal¬ 
function  correction.  Then,  the  modifications  to  the  Siegel -Wolf  model 
which  were  implemented  to  achieve  this  time  calculation  are  described. 
Finally,  the  methods  for  applying  the  deterministic  calculation  and  the 
computer  simulation  to  the  selected  sample  are  described. 
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EQUIPMENT  DESCRIPTION  AND  SAMPLE  SELECTION 


Description  of  Radio  Set  AN/URC-35 

As  described  in  the  technical  manual,  the  Radio  Set  AN/URC-35 
is  a  single  sideband,  high  frequency  set  designed  for  use  on  shipboard, 
vehicular,  and  shore  installation.  The  set  also  has  capabilities  for  gen¬ 
eral  purpose  use,  beach  landing  parties,  and  emergency  operations. 
Transmission  and  reception  on  280,000  channels,  spaced  100  cps  apart 
in  the  2  to  30  MC  range  are  provided.  Circuits  are  employed  for  auto¬ 
matic  digital  tuning  in  100  cps  increments.  The  equipment  includes 
vernier  control  for  continuous  tuning  between  1  KC  increments  to  permit 
compatibility  with  less  stable  transmitters.  Transmission  and  reception 
can  be  performed  in  upper  sideband  (USB),  lower  sideband  (LSB),  con¬ 
tinuous  ware  (CW),  and  c  ^mpatible  amplitude  modulation  (AM).  A  re¬ 
mote  radio  control  set  permits  transmission  and  reception  in  USB,  LSB, 
and  AM  from  a  remote  station. 

The  receiver -transmitter  accepts  audio  from  a  handset,  micro¬ 
phone,  or  coded  intelligence  from  a  CW  key  and  converts  it  to  one  of 
280,000  operating  frequencies.  In  all  transmitting  modes,  except  CW, 
voice  signals  modulate  a  500  KC  local  carrier.  The  resulting  double 
sideband  signal  is  filtered  according  to  the  operating  mode  and  con¬ 
verted  to  the  desired  RF  operating  frequency  by  a  triple  conversion 
process.  In  the  CW  mode,  the  500  KC  carrier  is  inverted  directly  into 
the  IF  amplifiers  at  a  coded  rate  and  then  processed  in  the  same  manner 
as  the  voice  signals.  The  frequency  standard  and  synthesizer  circuits 
of  the  receiver-transmitter  generate  the  500  KC  local  carrier  and  the 
injection  frequencies  required  for  the  IF  to  RF  and  RF  to  IF  conversions. 
In  addition,  the  500  KC  signal  is  used  as  the  carrier  reinsertion  signal 
during  single  sideband  reception. 

The  Radio  Set  AN/URC-35  consists  of  the  following  units:  Re¬ 
ceiver-Transmitter  RT-618/URC,  RF  Amplifier  AM-3007/URT,  An¬ 
tenna  Coupler  CU-937/UR,  Shock  Mount  MT-376 l/URC-35,  Whip  an¬ 
tenna  AT-1047/U,  Handset  H-169/Uwith  cord  and  plug  assembly  CK- 
1846A/U;  and  the  following  optional  items;  Power  Supply -Battery 
Charger  PP-4679/URC-35,  Remote  Radio  Control  Set  C-3697/URC, 
Microphone  M-109/U  with  cord  and  plug  assembly  CX-1846A/U,  CW 
Key,  Headset,  Rechargeable  28-volt  Battery  BB-421/U. 
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Selection  of  Failed  Subassemblies  aid  Components-- AN/URC-35 


To  achieve  a  sample  of  failures  which  is  representative  of  the 
failure  distribution  within  the  AN/URC-35,  equipment  reliability  and 
maintainability  data  were  employed.  These  data  were  developed  by 
the  manufacturer  of  the  Radio  Set  AN/URC-35,  General  Dynamics,  who 
performed  maintenance  task  analyses  for  the  Receiver-Transmitter 
unit  (RT-618/URC)  and  the  Radio  Frequency  Amplifier  (AM-3007/URT). 
Generail  Dynamics  also  developed  equipment  reliability  data  in  terms  of 
anticipated  mean  time  between  failure  and  failure  rate  (provided  in  per¬ 
centage  of  10,000  hours).  Mean  time  between  failure  was  given  to  the 
subassembly  level,  while  failure  rate  was  given  at  the  component  level. 

The  first  phase  of  the  sampling  procedure  involved  subassem¬ 
bly  selection  on  the  basis  of  mean  time  between  failure.  Subassemblies 
were  selected  so  that  a  representation  across  the  range  of  mean  time 
between  failure  was  acquired.  The  range  of  mean  time  between  fail¬ 
ure  given  by  General  Dynamics  for  the  28  subassemblies  in  the  AN/ 
URC-35  was  from  8,  136  hours  to  965,  251  hours,  with  a  mean  of  157,950 
hours  and  a  standard  deviation  of  248,617  hours.  Fifteen  subassem¬ 
blies  were  selected  for  inclusion  in  the  present  work  from  the  28  avail¬ 
able.  The  subassemblies  selected  were  distributed  around  the  mean  of 
157,  950  hours.  The  extremes  at  both  the  high  end  and  low  end  of  the 
mean  time  between  failure  distribution  were  not  selected  so  that  those 
subassemblies  finally  selected  tended  more  toward  the  mean  of  the  dis¬ 
tribution. 

A  frequency  distribution  of  the  time  between  failure  of  subas¬ 
semblies  was  constructed  on  the  basis  of  the  data  presented  by  General 
Dynamics.  This  frequency  distribution  appears  in  Table  7.  Table  7  in¬ 
dicates  that  20  subassemblies  fall  in  the  first  two  class  intervals,  from 
1,000  to  100,  000  hours  time  between  failure.  The  sample  of  subassem¬ 
blies  for  inclusion  in  the  present  study  was  stratified  to  reflect  this  dis¬ 
tribution.  The  predicted  time  between  failure  for  the  subassemblies  se¬ 
lected  for  consideration  in  the  present  work  is  also  presented  in  Table  7. 
Thus,  71  per  cent  of  the  subassemblies  (11)  in  the  sample  were  from  the 
first  two  class  intervals  of  Table  7  and  29  per  cent  (4  subassemblies) 
were  from  among  the  remaining  time  intervals. 
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Table  7 

Actual  and  Sample  Frequency  Distributions  of  Time  Between  Failure 
for  the  Receiver-Transmitter  and  the  Radio  Frequency  Amplifier  Units 


Time 

Between  Failures 
(in  thousands  of  hours) 


Frequency 


Sample 


950-999 
900-949 
850-899 
800-849 
750-799 
700-749 
650-699 
600-649 
550-599 
500-549 
450-499 
400-449 
350-399 
300-349 
250-299 
200-249 
150-199 
100-149 
50-  99 
1-  49 


Twenty-five  component  failures  were  selected  within  the  15  se¬ 
lected  subassemblies.  These  were  selected  primarily  on  the  basis  of 
component  failure  rate  data.  In  order  not  to  select  components  that 
required  very  frequent  repair/  replacement  or  very  infrequent  repair/ 
replacement,  the  mean  failure  rate  was  computed  for  all  the  compo¬ 
nents  in  a  selected  subassembly.  If  a  subassembly  contained  several 
component  types,  two  different  component  types  were  selected  from 
around  the  mean  failure  rate  for  all  the  components  in  the  subassembly. 
If  a  subassembly  contained  only  one  or  a  few  component  types,  a  com¬ 
ponent  failure  was  selected  in  such  a  manner  as  to  yield  a  variety  of 
different  electronic  component  failures  in  the  sample  and  to  represent 
the  continuum  of  predicted  malfunction  correction  time.  * 

Table  8  identifies  the  components  within  subassembly  so  selected 
as  well  as  the  failure  rate. 


Description  of  Radar  Set  AN/ APS- 115 

The  Radar  Set  AN/APS-115  is  an  airborne  radar  designed  for 
search  and  detection  of  surface  targets.  The  equipment  is  a  portion 
of  the  antisubmarine  warfare  system  of  the  P-3C  aircraft.  The  radar 
has  a  manually  tunable  transmitting  frequency  of  from  8. 5  to  9.  6  GHz, 
Other  characteristics  of  the  radar  set  include  its  60  MHz  frequency 
bandwidth,  143  KW  peak  power,  0.5  and  2.  5<j  sec  transmitter  pulse 
width,  400  Hz  line-locked  with  2.5  n  sec  pulse  width  and  1600  Hz  line- 
locked  with  0.5 n  sec  pulse  repetition  frequency,  6  rpm  with  2.5/j  sec 
pulse  width  and  12  rpm  with  0.5  fi  sec  pulse  width  antenna  scan  speed, 
antenna  scan  modes  of  searchlight,  45°  sector,  and  360°  scan,  manual 
tilt  coverage  of  +  10  to  -20  degrees,  and  pitch  and  roll  tilt  stabilization. 
The  complete  radar  set  is  composed  of  the  following  units:  Antenna  Po¬ 
sition  Programmer  MX-7930,  Receiver/Transmitter  RT-889,  Antenna 
AS-2146,  Control  Antenna  C-7511,  and  Control  Radar  Set  C-7512. 


*Such  data  were 
formed  by  the  General 


provided  as  a  part  of  the  task  analyses  per- 
Dynamics  Corporation. 
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Table  8 


Sample  of  Components  for  Technician  Reliability  Analysis 
_  of  AN/  URC-35 _ 


Subassembly* 

Component 

F ailure  Rate 
(per  cent  per 
10, 000  hours) 

Receiver-Transmitter  RT-618/ URC 

1.  Filter  Box 

1.  Capacitor 

.  344 

Filter  Box 

2.  Filter 

.  350 

2.  Mode  Selector 

3.  500  KC  Vmplifier 

2.  683 

3.  Receiver  IF  Audio 

4.  Connector 

.  331 

Receiver  IF  Audio 

5.  Amplitude  Modulation  De 

lector  3.745 

4.  Transmit  Audio 

6.  Transformer 

.213 

Transmit  Audio 

7.  Printed  Circuit  Board 

.  200 

5.  RF  Amplifier 

8.  Megacycle  Assembly 

.  699 

RF  Amplifier 

9.  RF  Amplifier  Assembly 

1.  023 

6.  Frequency  Standard 

10.  Oven  Housing 

.310 

7.  Noise  Blanker 

11.  Connector 

.  052 

Noise  Blanker 

12.  Blanker  Board 

9.  943 

8.  1  &  10  KC  Synthesizer 

13.  10  KC  Switch 

.  400 

1  &  10  KC  Synthesizer 

14.  Output  &  Blanker 

2.728 

9.  Spectrum  Generator 

15.  100  KC  Spectrum 

11.367 

Spectrum  Generator 

16.  Pulse  Inverter 

.  856 

10.  RF  Translator 

17.  Cordwood  Module 

7.  177 

11.  Code  Generator 

18.  Code  Generatoi 

2.  365 

12.  20  v.  Regulator 

19.  20  v.  Regulator 

18.  938 

13.  Switch  Assembly 

20.  Resiotor  Variable 

.  949 

Switch  Assembly 

21.  Switch  Section 

.  600 

Radio  Frequency  Amplifier 

14.  Filter  Box 

22.  Capacitor  Feedthrough 

.719 

Filter  Boa 

23.  Capacitor 

2.  908 

15.  Interlock  Switch  Assembly 

24.  Interlock  Switch 

1.  200 

16.  RF  Board 

25.  RF  Board 

1.  628 

*  A  sixteenth  subassembly  \\as  added  to  provide  a  greater  range  of 
components  in  the  final  sample. 
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Selection  of  Failures- -AN/APS- 115 

The  basic  source  for  selecting  the  sample  of  AN/ APS- 115  fail¬ 
ures  for  inclusion  in  the  present  study  was  Fleet  collected  time  between 
removals  data  (the  ratio  of  removals  of  faulted  components  to  unit  fly¬ 
ing  hours).  Maintainability  records  for  the  AN/APS-115  covering  the 
13  months  from  January  1970  to  February  1971  were  made  available  by 
the  group  which  compiled  the  data  at  NAS,  Patuxent  River.  Only  organi¬ 
zational  level  repairs  were  considered  for  inclusion  in  the  sample.  Dur¬ 
ing  this  period,  29  component  removals  and  replacements  occurred  at 
the  organizational  level.  The  range  of  time  between  removals  was  from 
606  to  19,  392  hours,  with  a  mean  of  9,  375  hours  and  a  standard  deviation 
of  6,  771  hours. 

A  frequency  distribution  of  the  time  between  removals  appears  in 
Table  9.  The  final  sample,  presented  in  the  third  column  of  Table  9,  con¬ 
sisted  of  24  failed  components  selected  so  as  to  reflect  the  distribution 
which  appears  in  the  second  column  of  Table  9.  Table  10  lists  these  24 
components.  The  sample  of  24  components  had  a  mean  time  between  re¬ 
movals  of  8,442  and  a  standard  deviation  of  6,  271. 
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Table  9 


Frequency  Distribution  of  Time  Between  Component 
Removals  for  the  AN/ APS- 115  and  for  Sample 


TBR 

(in  hundreds 
of  hours) 


Fleet 

Frequency 


Sample 
Frequency 


190-199 

180-189 

170-179 

160-169 

150-159 

140-149 


130-139 

120-129  3 

110-119 

100-109 

90-  99  6 

80-  89 

70-  79  1 

60-  69  1 

50-  59 

40-  49  2 

30-  39 

20-  29  4 

10-  19  3 

1-9  2 


29 


4 


3 


5 

1 

1 

2 

4 

2 

_2 

24 
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Table  10 


Sample  of  Components  for  Technician  Reliability  Analysis 
_ of  AN/ APS- 115 _ 


Nomenclature 


Mean  Time 
Between  Removals 


AZ  Amp  Assembly 

12,928 

PS  Logic  Assembly 

6,464 

P-M  Logic  Assembly 

4,848 

MSSW  Logic  Assembly 

19,392 

AS  Synchronizer 

19,392 

BITE  Logic  Assembly 

9,696 

Tilt  Function  Generator  Assembly 

9,696 

15  V  Power  Supply  Assembly 

19,392 

AZ  Scan  Programmer  Assembly 

970 

AFC  Assembly 

2,770 

IF  Amp  Assembly 

2,424 

Solid  State  Oscillator 

19,392 

Transmitter  Assembly 

606 

Thyratron  Trigger-Regulator  Assembly 

2,041 

Logic,  Isolation  Amplifier 

12,928 

BITE  No.  2 

9,696 

20  V  Power  Supply 

9,696 

Pressurization  Unit  Assembly 

2,586 

Antenna  Assembly 

4,848 

Gear  Box  3/8  Azimuth 

616 

Gear  Box,  Elevation 

1,847 

Antenna  Control  Unit  4 

9,696 

Control  Radar  Unit 

7,757 

Tilt,  Amplifier  Assembly 

12,928 
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AUGMENTATION  OF  COMPUTER  SIMULATION  MODEL 


To  augment  the  Siegel -Wolf  simulation  model  so  that  its  out¬ 
put  would  be  based  on  the  same  input  data  substrate  as  the  determin¬ 
istic  calculations  of  the  probability  of  malfunction  correction  success, 
a  number  of  changes  were  implemented  in  the  computer  routine. 

All  changes  made  to  the  basic  simulation  model,  described  in 
Chapter  I,  were  made  external  to  the  basic  mission  simulation  pro¬ 
gram.  Progi  ammatically,  these  changes  were  implemented  through 
an  independent  subroutine  called  RAM  (Reliability  and  Maintainability) 
and  through  changes  to  e  INPUT  routine.  Thus,  all  changes  to  the 
simulation  model  const  fe  a  new  preprocessing  of  the  basic  probabil¬ 
ity  (py)  input  data  to  refit,  improved  personnel  performance  success/ 
failure  data.  Following  the  preprocessing,  the  adjusted  probability  data 
are  used  by  the  original  model,  and  the  original  program  is  essentially 
unmodified. 

The  decision  to  preprocess  the  input  data,  rather  than  modify 
the  model  per  se,  is  basic  to  the  approach  to  the  design  of  the  simula¬ 
tion  here  considered.  The  alternative  was  a  recalculation  of  conditions 
at  the  time  of  simulation  of  each  subtask.  The  preprocessing  approach 
was  selected  for  the  following  reasons: 


1.  It  provided  for  adjustments  to  be  made  once  fox- 
each  N  iteration  simulations,  rather  than  perform- 
mance  of  these  calculations  for  each  of  N  itera¬ 
tions.  This  reduces  the  computer  time  required 
for  simulation. 

2.  It  required  only  small  changes  to  the  original  com¬ 
puter  program  for  the  simulation,  those  changes 
being  restricted  to  the  INPUT  routine  and  formats. 
This  reduced  program  development  and  checkout 
times. 

3.  It  promised  comparable  accuracy  and  validity  of 
results  within  the  context  of  Monte  Carlo  simula¬ 
tion  methods. 

4.  It  left  the  original  model  essentially  unaltered. 
Thus,  its  internal  validity  and  operating  charac¬ 
teristics  remain  known  and  unaffected. 
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Input  Data  Required 

The  input  data  required  by  the  RAM  preprocessor  are  essen¬ 
tially  the  same  as  those  required  by  the  basic  Siegel -Wolf  model  ex¬ 
cept  for  the  addition  of  two  one-digit  code  fields  in  the  input  punched 
cards.  The  first  of  these  new  inputs,  the  Job  Activity  Group  (JAG 
code)  is  punched  into  card  column  76  of  the  card  for  every  task  ele¬ 
ment  performed  by  any  technician.  The  code  assignments  for  the  9 
active  job  activity  (factor)  groups  are  given  in  Table  11.  That  is, 
each  task  element  is  categorized  during  the  task  analysis  as  belong¬ 
ing  to  one  and  only  one  of  the  factors.  The  code  assignments  are  also 
shown  in  Table  11. 

The  second  one-digit  code  is  contained  in  the  operator  cards 
as  an  addition  to  data  previously  punched  in  these  cards.  This  digit 
represents  the  Job  Operator  Type  (JOT)  punched  in  card  column  42 
to  identify  the  Navy  rate  of  each  of  the  technicians  simulated.  The 
code  assignment  is  shown  in  Table  12.  This  code  controls  the  selec¬ 
tion  of  appropriate  success  probability  factors,  as  described  later, 
for  the  type  of  technician  (Navy  rate)  selected  for  simulation. 
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Table  11 


Job  Activity  (Factor)  Group  (JAG)  Codes 


Code 

Description 

0 

None  of  the  following 

1 

El  e  ct  r  o  -  c  og  nitio  n 

EC 

2 

Electro-repair 

ER 

3 

Instruction 

I 

4 

Electro-safety 

ES 

5 

Personnel  Relationships 

PR 

6  . 

Electronic  Circuit  Analysis 

ECA 

7  . 

Equipment  Operation 

EO 

8 

Using  Reference  Material 

URM 

9 

Inspection 

El 

Table  12 

Job  Operator  Type  (JOT)  Codes 

Code  Description 

0  None  of  the  following 

1  Electrician's  Mate 

2  Electronic  Technician 

3  Fire  Control  Technician 

4  Interior  Communications  Technician 

5  Radar  Technician 

6  Radioman 

7  Sonar  Technician 

8  Torpedoman 


EM 

ET 

FT 

IC 

RD 

RM 

ST 

TM 
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Flow  Logic  Description 

A  description  of  the  logic  and  arithmetic  operations  perform¬ 
ed  by  the  preprocessor  is  given  in  this  section.  It  is  noted  that  the 
use  of  this  preprocessor  is  designed  to  apply  exclusively  to  mainte¬ 
nance  and  repair  actions  due  to  the  unique  tailoring  of  the  processing 
to  data  available  for  Naval  personnel  performing  these  specific  types 
of  duties. 

The  function  of  this  preprocessing  is  to  adjust  the  data_usually 
provided  to  the  model  as  task  element  success  probabilities  (p^  values) 
by  type  of  technician  (JOT)  and  by  type  of  job  activity  (factor )( JAG). 

The  sequence  of  operation  is  given  in  the  flow  chart  presented 
as  Figure  4.  It  is  assumed  that  all  input  data  are  prepared  in  card 
form  as  usually  provided  for  the  model  plus  the  two  additional  codes 
discussed  above.  This  general  flow  chart  was  implemented  program¬ 
matically  by  the  sequence  of  operations  shown  in  the  more  detailed 
flow  chart  presented  as  Figure  5. 

Each  of  the  technicians  to  be  simulated  is  preprocessed  in¬ 
dependently.  If  no  JOT  is  given  for  either  operator,  then  the  entire 
preprocessor  is  bypassed  and  processing  continues  with  the  model 
directly.  If  a  JOT  from  1  to  8  is  given  for  a  technician,  the  pro¬ 
cessing  is  performed  for  each  operator  for  which  a  non-zero  JOT 
is  given.  The  processing  for  each  technician  proceeds  as  follows  for 
each  of  the  JAG  codes. 

For  a  JAG  of  zero,  no  adjustment  is  made  in  subtask  prob¬ 
ability  (pij)  values.  When  JAG  is  other  than  zero,  first  the  number 
of  subtasks  in  each  JAG  is  determined  and  the  product  of  the  pjj  values 
(PG)in  each  JAG  is  calculated.  In  this  and  all  preprocessor  operations, 
decision  subtasks  are  not  utilized  as  the  probability  associated  with 
them  does  not  represent  subtask  success  and  is  therefore  not  subject  to 
modification  in  this  process.  This  product  represents  the  cumulative 
probability  for  all  operator  actions  in  each  JAG.  For  example,  if 
there  are  five  subtasks  to  be  performed  by  a  technician  in  a  repair 
task  and  all  have  JAG  =  4  (electrosafety)  then  the  product  of  these  five 
py  values  represents  the  likelihood  of  that  operator  successfully  per¬ 
forming  all  electrosafety  jobs  required  by  the  malfunction  correction. 
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Figure  5.  Basic  programming  tlamants  for  preprocessor. 

55 


It  is  this  product  which  ideally  should  equal  the  personnel  factor  re¬ 
liability  values  described  in  Chapter  I  and  shown  in  Figure  2.  These 
are  called  success  probability  of  the  group  (SPOG).  It  is  the  basic 
purpose  of  this  preprocessor  to  adjust,  if  necessary,  each  input  py 
value  (by  JAG)  so  that  this  agreement  occurs  within  a  reasonable  toler¬ 
ance.  The  tolerance  was  selected  to  be  0.  01.  Thus,  if  the  product 
|PG  -  SPOG(JAG,  JOT)  |  <  0.  01,  then  no  further  processing  is  required 
and  the  next  JAG  is  considered.  In  the  case  that  only  one  subtask  ap¬ 
pears  in  a  given  JAG,  the  p^j  of  the  subtask  is  set  equal  to  the  SPOG 
value  for  the  JAG  and  JOT  under  consideration. 

In  the  event  that  further  processing  is  required,  the  NGth  root 
of  SPOG  is  calculated  where  NG  is  the  number  of  task  elements  having 
a  given  JAG.  This  root  represents  an  initial  value  for  p^  which  satis¬ 
fies  the  desired  condition.  That  is,  if  all  task  elements  possess  an 
equal  p^.,  then  their  product  would  equal  the  appropriate  SPOG  value. 
Consequently,  this  root,  called  TNR,  is  an  initial  value  used  in  the  de¬ 
termination  of  individual  new  p-  values. 

_  A  value  is  then  calculated  for  the  average  increase  or  decrease 
in  py  for  individual  task  elements.  This  value  is 

NG 

AID  =  .S^TNR  -  Fij). 

The  total  potential  increase  for  all  task  elements  of  the  group  is  then 
calculated: 

NG 

TP  *.2.(1“?..). 

1=1  rlj 
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New  success  probabilities  <or  each  task  element  in  the  group 
are  then  calculated  according  to  ths  formula: 


new  p..  =  old  Plj  +  U  -  PyHp  • 

This  represents  a  change  to  the  original  p^j  value  which  is  a  function 
of  the  potential  increase  of  the  individual  subtask  (1  -  p..)  and  the 
ratio  of  average  to  total  possible  change. 

To  verify  that  this  process  has  given  pij  values  which  meet 
the  SPOG  criterion,  the  product  PG  is  again  calculated  and  the  test 
|PG  -  SPOG  |  <0.01  is  again  applied.  If  passed,  processing  proceeds 
to  the  next  JAG;  otherwise  another  iteration  is  made  of  the  py  adjust¬ 
ment  algorithm. 

In  tracing  through  the  task  analysis  in  order  to  determine  the 
number  of  subtasks  of  each  type,  the  computer  follows  the  most  prob¬ 
able  success  failure  probabilites,  (i,j)s  and  (i,  j)f.  Those  task  elements 
which  are  not  included  along  that  path  are  therefore  not  modified  in  p^j. 

A  computational  example  is  presented  as  Figure  6. 
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INPUT 

JAG:  ES,  group  4 

JOT:  IC,  operator  type  4 
SPOG:  0.65 

Task  elements 
with  JAG  =  4 

pii 

new  pjj 
(see  below) 

1 

0.85 

0.947 

4 

0.98 

0.993 

5 

0.75 

0.912 

6 

0.90 

0.965 

7 

0.75 

0.912 

8 

0.99 

0.996 

9 

0.90 

0.965 

18 

0.75 

0.912 

NG  =  8 


i i v«  i  i 

PG  =  IT  p. .  =  0.  85  x  0.  G8  . . .  x  0.  75  =  0.  320;  0.  65  -  0.  32  <  0.  01? 

I  iJ 

TNR  =  8Vo.  65  =  0.95 


NG  _ 

TP  «=  S  (1  -  p. .)  =  0.  15  +  0.  02  +  . . .  +  0.  25  =  1.  13 
i=l  ^ 

NG  _ 

AID  =  S  (TNR  -  p. .)  =  0.  10  -  0.  03  +  . . .  +  0.  2  =  0.  73 
i=l 


’ll 


new 


41  = 


probabilities/ 


18, 1  = 


0.  85  +  (0. 

0.  98  +  (0.02)(T^|) 
0.75  +  (0.  25)<y~) 


0.947 

0.  993 


0.912 


Figure  6.  Example  of  preprocessing  calculation 
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Recorded  Preprocessor  Output 


Prior  to  adjustment  of  task  element  success  probabilities,  a 
short  table  of  data  is  recorded  for  printing  by  the  preprocessor.  It 
includes  for  each  job  activity  (JA)  the  number  of  task  elements  in  the 
group  (NG),  the  product  of  group  p1^  values  (PG),  and  the  target  suc¬ 
cess  probability  of  the  group  (SP).  Groups  for  which  no  adjustment 
in  pjj  values  need  be  made  are  identified  by  OK  =  1;  others  by  OK  =  0. 
A  sample  of  this  output  is  given  in  Figure  7. 

Following  the  preprocessing,  a  table  is  recorded  for  printing 
(Figure  8)  for  each  operator.  It  contains  for  each  task  element  the 
task  element  number,  the  JAG  (GROUP),  the  number  of  task  elements 
in  the  group  (N-GRP),  and  both  the  old  and  new  task  element  probabil¬ 
ities  (OLDPROB  and  NEWPROB). 


Frequency  Distribution  Augmentation 

Because  of  the  importance  in  this  research  of  frequency  distri¬ 
bution  plots  of  the  amount  of  time  used  by  the  simulated  operator,  the 
Siegel -Wolf  simulation  model  was  further  augmented  w  ith  a  subroutine 
to  record  and  plot  such  frequency  distributions. 
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Figure  7.  Example  of  initial  preprocessor  output, 
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Figure  8. 

Sample*  final  preprocessor 
output. 
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TECHNIQUE  APPLICATION 


Having  selected  two  current  Navy  electronic  systems  and  a 
representative  sample  of  malfunction  corrections  within  these  sys¬ 
tems  and  having  developed  and  tested  the  prepx  ocessing  subroutine 
to  the  Siegel -Wolf  model,  the  deterministic  technique  (for  predicting 
the  probability  of  successful  malfunction  correction)  and  the  computer 
simulation  (for  predicting  time  to  complete  successfully  the  correc¬ 
tion)  were  applied. 


Task  Analyses 


In  order  to  obtain  the  basic  data  required  for  application  of 
these  techniques,  complete  task  analyses  were  completed  for  all  mal¬ 
function  corrections  included  in  the  samples.  These  task  analyses 
were  first  developed  with  the  assistance  of  Fleet  personnel  at  the  Nav¬ 
al  Ships  Engineering  Center,  Norfolk  Division  (AN/URC-35)  and  at 
NAMTD,  NAS  Patuxent  River  (AN/ APS- 1 15),  who  possess  the  responsi¬ 
bility  for  maintaining  these  equipments. 

Having  developed  and  set  the  task  analyses  in  proper  preliminary 
form,  the  task  analyses  were  resubmitted  to  the  Fleet  personnel  for  re¬ 
view.  Revisions  and  modifications  suggested  by  these  persons  were  in¬ 
corporated  into  the  analyses.  In  all,  49  such  analyses  were  completed. 
A  sample  of  one  such  task  analysis  is  presented  in  Table  13. 
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Sample  Task  Analysis 

Task:  Repair  Filter--AN/URC-35 

1.  Opens  drawer  (AM3007)  29.  Removes  old 
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Probability  of  Successful  Malfunction  Correction  Determination 


To  apply  the  mathematical  technique  for  predicting  malfunc¬ 
tion  correction  success  probability,  an  analyst,  who  was  thoroughly 
familiar  with  the  malfunction  correction  procedure  and  with  the  defi¬ 
nitions  of  the  job  factors,  reviewed  each  task  analytic  statement  to 
determine  which  job  factor(s)  are  applicable  to  each  of  the  49  mal¬ 
function  corrections.  An  example  of  the  result  (for  the  task  analysis 
shown  in  Table  13)  is  presented  below: 


Series  Case:  Pr(s]  =  RECRERREORURM  = 

Parallel:  Pr[s]  -  [1  -  (1-REC)2][1  -  ( 1 -RER)2][1 -( 1 -REO>2] 

[i  -  ] 


These  data  were  then  compounded  in  accordance  with  the  meth¬ 
ods  described  in  Chapter  I  of  this  report  to  yield  separate  series  and 
parallel  malfunction  correction  success  probability  predictions  for  each 
of  the  49  malfunctions  in  the  samples.  In  all  cases,  the  malfunction 
correction  was  assumed  to  be  performed  by  persons  in  the  electronics 
technician  rate,  the  Navy  rate  which  possesses  the  responsibility  for 
maintaining  the  AN/URC-35  and  the  AN/ APS- 115  equipments. 


Time  for  Malfunction  Correction  Determination 


For  10  of  the  AN/URC-35  malfunction  corrections  and  for  five 
of  the  AN/ APS- 115  malfunction  corrections  the  input  data,  required 
by  the  Siegel-Wolf  model  and  the  preprocessor  developed  in  the  course 
of  the  present  work,  were  prepared.  Computer  simulation  of  the  per¬ 
formance  of  these  15  malfunction  corrections  was  performed  to  yield 
estimates  of  malfunction  correction  time. 
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To  select  the  malfunction  corrections  to  be  involved  in  the  com¬ 
puter  simulations,  a  set  was  selected  for  the  AN/URC-35  so  as  to  re¬ 
flect  the  MTTR  range  and  mean.  The  AN/ APS- 115  sample  for  com¬ 
puter  simulation  was  selected  on  the  basis  of  the  MTBR  range  and  mean 
for  this  equipment.  The  malfunction  corrections  computer  simulated 
were: 


AN/  URC-35 

1.  Filter 

2.  Connector  on  Receiver  IF  Audio 

3.  Transformer 

4.  RF  Amplifier  Assembly 

5.  Megacycle  Assembly 

6.  Connector  on  Noise  Blanker 

7.  Switch  10  KC 

8.  Code  Generator 

9.  Interlock  Switch 

10.  RF  Board 

AN/ APS- 115 

1.  Azimuth  Servo  Amplifier  Assembly 

2.  AS  Synchronizer 

3.  Automatic  Frequency  Control  Assembly 

4.  Thyraton  Trigger-Regulator  Assembly 

5.  20  V  Power  Supply 

The  Siegel -Wolf  model  allows  for  simulation  of  persons  who 
work  at  various  speeds  (also  interpreted  as  persons  at  different  pro¬ 
ficiency  levels).  In  the  case  of  a  single  technician  being  assigned  to 
the  malfunction  correction,  the  speed  parameter  (Fj)  was  varied  over 
three  levels.  This  variation  allowed  the  determination  of  time  pre¬ 
dictions  in  the  unitechnician  case  for  average  speed  (proficiency)  tech¬ 
nicians  (Fj  =  1.0),  for  "fast"  technicians  (F^  =  0.9),  and  for  "slow" 
technicians  (Fj  =  1. 1).  In  all  cases,  the  stress  threshold  parameter 
in  the  model  was  set  at  2.3.  This  value  was  previously  found  to  repre¬ 
sent  an  average  value  for  Naval  technicians. 
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For  those  computer  simulations  involving  two  technicians  as¬ 
signed  to  the  same  malfunction  correction  (parallel  case),  the  g 
values  assigned  to  the  teams  were:Fji2  =  0.9.  0.9;  Fji  2  =  1.  u,  *1.0; 
and  Fj^  2  =  1*  1.  This  parametric  variation  allowed  for  predic¬ 

tions  of  time  for  successful  malfunction  correction  for  teams  com¬ 
posed  respectively  of  two  "fast,  "  two  "average,  "  or  two  "slow"  tech¬ 
nicians. 

Since  the  Siegel-Wolf  simulation  model  is  stochastic  in  nature, 
a  number  of  simulations  (iterations)  must  be  performed.  Each  itera¬ 
tion  will  yield  a  different  output  (malfunction  correction  time).  In  the 
present  case,  100  simulations  were  performed  for  each  malfunction 
correction  and  for  each  operator  type  (Fj  value)  or  team  composition 
involved.  The  resultant  data  are  held  to  represent  the  predicted  mal¬ 
function  correction  time  distribution  for  each  malfunction  and  for  each 
operator  type  or  team  composition. 


Interuser  Reliability 

Since  the  deterministic  technique  depends  on  analyst  assignment 
of  relevant  job  factors,  a  subinvestigation  was  performed  into  the  ex¬ 
tent  of  interuser  agreement  to  be  anticipated  from  such  factor  assign¬ 
ments.  Ib  this  end,  three  analysts  independently  made  job  factor  as¬ 
signments  to  each  of  the  49  malfunction  corrections  here  involved. 

These  assignments  were  made  on  the  basis  of  the  task  analyses,  de¬ 
scribed  above.  Phi  coefficients  were  then  calculated  to  represent  the 
extent  of  agreement  between  analysts  and  the  overall  agreement  among 
the  three  analysts.  This  latter  determination  was  based  on  a  procedure 
suggested  by  Guilford  (1965,  p.  334). 
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CHAPTER  HI 


RESULTS 


The  computational  technique,  described  in  Chapter  I  of  the  cur¬ 
rent  report,  was  applied  to  all  25  malfunction  corrections  sampled  for 
the  URC-35  radio  set.  The  probability  of  success  prediction  for  a  given 
malfunction  correction  involves  the  multiplicative  relationship  among 
all  job  factors  involved  in  that  malfunction  correction.  For  example,  if 
the  foUowing  job  factors  were  involved  for  a  given  malfunction  correction: 
electro-cognition,  electro-repair,  equipment  operation,  and  using  refer¬ 
ence  material,  the  formula  for  the  probability  of  task  success  in  the  se¬ 
ries  (single  technician)  case  is: 

Pr^  =  RECRERREORURM 


since  each  must  be  performed  satisfactorily  for  successful  task  perform¬ 
ance.  For  the  parallel  case  (two  technicians  assigned  to  the  malfunction 
correction)  the  formula  is; 

Pr[s] .  [i  -  (1-REC)2][1  -  <i-Rer)2][>  -  »-RE0>2][i  -  <1-RURM)21 

The  predicted  probability  that  a  maintenance  technician  (in  the 
electronics  technician  rating)  will  satisfactorily  complete  each  malfunc¬ 
tion  correction  on  his  own,  and  the  parallel  personnel  reliabilities  are 
presented  in  Table  14.  For  the  single  technician,  the  range  of  predicted 
probability  for  successful  malfunction  correction  on  the  first  attempt  is 
from  .66  to  .90.  The  parallel  technician  situation  indicates  a  consider¬ 
able  increase  in  success  probability.  Here,  the  range  is  from  .  95  to  .  98. 
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Table  14 


Technician  Reliability  Predictions  for  Sample  of  Malfunction 
_ Corrections  for  the  Radio  Set  AN/URC-35 _ 


Malfunction 

Correction 

Number 

One 

Technician 

Two 

Technicians 

1 

.73 

.96 

2 

.71 

.95 

3 

.74 

.97 

4 

.74 

.97 

5 

.74 

.97 

6 

.72 

.96 

7 

.72 

.96 

8 

.74 

.96 

9 

.74 

.97 

10 

.74 

.97 

11 

.74 

.97 

12 

.74 

.97 

13 

.74 

.97 

14 

.74 

.97 

15 

.74 

.97 

16 

.74 

.97 

17 

.74 

.97 

18 

.74 

.97 

19 

.74 

.97 

20 

.73 

.96 

21 

.90 

.98 

22 

.77 

.96 

23 

.79 

.97 

24 

.72 

.96 

25 

.66 

.96 
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The  probability  that  a  maintenance  technician  in  the  electronics 
technician  rating  will  satisfactorily  complete  any  random  one  of  the  25 
AN/URC-35  malfunction  corrections  sampled  may  be  expressed  by  a 
measure  of  central  tendency.  In  this  instance,  the  median  and  mean 
result  in  the  same  probability  value,  .  74.  The  probability  that  two 
maintenance  technicians  in  the  electronics  rating  working  together  will 
satisfactorily  complete  any  random  one  of  the  25  malfunction  correc¬ 
tions  on  the  first  attempt  is  .  97. 

Reliability  Estimations  for  the  AN/APS-115 

The  probabilities  of  successful  malfunction  correction  (techni¬ 
cian  reliabilities)  were  calculated  in  the  same  manner  for  the  AN/APS- 
115  as  for  the  AN/URC-35.  The  technician  reliabilities  for  the  24  mal¬ 
functions  sampled  in  the  APS- 115  are  presented  in  Table  15. 

The  APS-115  is  a  largely  modularized  electronic  system,  as  is 
the  URC-35.  In  a  modularized  system,  and  at  the  organizational  level 
of  repair,  the  correction  of  a  traced  malfunction  is  often  the  simple  re¬ 
placement  of  the  entire  module.  This  has  the  effect  of  inducing  a  fair 
degree  of  similarity  in  the  job  factors  involved  in  various  malfunction 
corrections.  The  result  in  terms  of  the  present  calculations  is  a  group¬ 
ing  of  the  probability  values  into  approximately  six  classes. 

For  the  single  technician  situation,  the  probability  of  successful 
malfunction  correction  on  the  first  attempt  ranged  from  .  50  to  .63.  This 
range  shifts  upward  for  two  technicians  working  together.  For  the  par¬ 
allel  situation  the  range  is  .  90  to  .92,  The  probability  that  any  random 
one  of  the  24  malfunctions  can  be  corrected  satisfactorily  on  the  first  at¬ 
tempt  is  .56  for  the  electronics  technician  working  alone.  For  a  pair  of 
electronics  technicians  working  together  the  probability  of  satisfactory 
completion  is  .91. 
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Table  15 


Technician  Reliability  Predictions  for  Sample  of  Malfunction 
Corrections  for  the  Radar  Set  AN/ APS- 115 _ 


Malfunction 

Correction 

Number 

One 

Technician 

Two 

Technicians 

1 

.56 

.91 

2 

.  63 

.  92 

3 

.  56 

.91 

4 

.  56 

.91 

5 

.  56 

.91 

6 

.  56 

.91 

7 

.  56 

.91 

8 

.56 

.91 

9 

.  56 

.91 

10 

.63 

.92 

11 

.  50 

.  90 

12 

.51 

.91 

13 

.56 

.91 

14 

.56 

.91 

15 

.50 

.90 

16 

.56 

.91 

17 

.56 

.91 

18 

.63 

.92 

19 

.50 

.90 

20 

.56 

.91 

21 

.  56 

.91 

22 

.  50 

.  90 

23 

.  56 

.91 

24 

.  56 

.91 
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TIME  FOR  MALFUNCTION  CORRECTION 


As  stated  in  Chapter  I,  the  logic  of  the  present  method  for  de¬ 
termining  human  reliability  rests  on  two  separate  analytic  methods: 

(1)  a  deterministic  calculation  of  the  probability  of  successful  mal¬ 
function  correction, and  (2)  a  computer  simulation  in  order  to  determine 
the  predicted  time  for  successfully  completing  the  malfunction  correc¬ 
tion.  The  results  from  the  application  of  the  deterministic  method  were 
presented  above.  The  present  section  of  this  report  presents  the  mal¬ 
function  correction  time  predictions,  as  derived  from  application  of  the 
modifications  to  the  Siegel -Wolf  stochastic  simulation  model.  The  mod¬ 
ifications  of  the  model,  to  make  the  model  compatible  with  the  determin¬ 
istic  calculations,  were  described  in  Chapter  II. 

Series  Case--AN/URC-35 

The  predicted  time  [for  the  series  case  (single  repair  technician)] 
to  repair  the  10  AN/URC-35  malfunctions  included  in  the  malfunction 
correction  subsample  subjected  to  computer  simulation  is  presented  in 
Figures  9  to  11.  For  each  malfunction  represented  in  Figures  9  to  11, 
the  cumulative  percentage  of  repairs  (ordinate)  within  a  given  time  peri¬ 
od  (abscissa)  is  given.  In  each  presentation,  separate  plots  are  given 
for  the  "fast"  and  for  the  "slow"  simulated  technician.  For  purposes 
of  presentational  clarity,  the  plot  for  the  "average"  technician  is  not 
given.  Such  a  plot  may  easily  be  interpolated  between  the  plots  for  the 
"fast"  and  the  "slow"  technician.* 

Thus,  for  malfunction  correction  2  (replace  filter)  and  a  "fast" 
technician,  50  per  cent  of  the  corrections  of  this  malfunction  are  pre¬ 
dicted  to  be  performed  successfully  in  2280  seconds  (38  minutes).  For 
a  "slow"  technician,  50  per  cent  of  the  corrections  of  this  malfunction 
are  predicted  to  be  completed  in  2340  seconds  (39  minutes).  The  range 
of  predicted  time  for  the  "fast"  technician  is  from  1925  seconds  (32.  1 
minutes)  to  2425  seconds  (40.4  minutes).  The  range  of  predicted  time 
for  successful  malfunction  correction  in  the  case  of  the  "slow"  technician 
is  from  2145  seconds  (35.  8  minutes)  to  2425  seconds  (40.  4  minutes). 

The  predicted  probability  of  successful  malfunction  repair  within  these 
ranges  of  time,  as  stated  earlier,  is  .74. 


*  "Fast"  technician:  F.  =  0.  9 
"Slow "  technician:  Fj  =  1.  1 
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.  Predicted  malfunction  correction  time  (single  technician)  for  malfunctions 
2,4,6  and  9  (AN/URC  35). 


MALFUNCTION  CORRECTIONS 


Predicted  malfunction  correction  time  (single  technician)  for  malfunctions 
10,11,13  and  18  (AN/URC-35). 
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The  remaining  plots  in  Figures  9  to  11  may  be  similarly  inter¬ 


preted. 


If  one  considers  all  10  malfunctions  in  the  sample,  the  minimum 
predicted  time  for  malfunction  correction  in  the  AN/URC-35  appears  to 
be  1075  seconds  (17.9  minutes)(malfunction  4,  replace  receiver  IF  audio 
module)  and  the  maximum  time  appears  to  be  2625  seconds  (43.  8  minutes) 
(malfunction  25,  replace  RF  board).  The  range  of  median  values  over 
the  10  simulated  malfunction  corrections  is  summarized  below: 


"fast"  technician 
"average"  technician 
"slow"  technician 
overall 


1285  to  2500  seconds 
1303  to  2535  seconds  (interpolated) 
1325  to  2570  seconds 
1969  seconds 


Accordingly,  for  the  single  technician  case  and  on  the  basis  of 
these  results,  the  methods  predict  that:  (1)  the  median  time  to  repair 
the  AN/URC-35  radio  set  is  between  21.  4  and  42.  8  minutes,  (2)  the 
overall  median  is  32.  8  minutes,  and  (3)  the  mean  probability  of  accom¬ 
plishing  a  .  epair  within  this  time  interval  is  .  74. 

V/e  lote  that,  the  results  suggest  greater  time  variation  within 
operator  speed  (proficiency,  Fj)  values  than  between  speed  values. 

Since  most  of  the  trouble  shooting  on  this  equipment  unit  is  performed 
with  the  use  of  simple  test  equipment  and  since  most  of  the  repair  ac¬ 
tions  consist  of  module  replacement,  such  a  result  might  have  been 
anticipated.  With  such  an  equipment  design,  malfunction  correction  is 
reduced  to  a  least  common  denominator,  and  differences  in  maintenance 
technician  proficiency  may  receive  little  opportunity  to  manifest  them¬ 
selves.  Possibly,  only  with  more  sophisticated  and  demanding  equip¬ 
ment  repair  actions  would  such  differences  become  manifest.  Moreover, 
we  note  that  even  for  the  Fj  =  1.  1  operator,  basic  electronic  training  and 
training  on  the  equipment  involved  is  assumed.  This  interpretation  has 
been  placed  on  the  predictions  generated  by  the  Siegel -Wolf  model  in 
prior  employments  and  continues  to  hold  in  the  present  case. 
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Parallel  Case--AN/URC-35 


A  similar  set  of  analyses  was  conducted  from  the  situation  in 
which  two  maintenance  technicians  are  assigned  to  complete  each  mal¬ 
function  correction.  This  is  the  case  of  parallel  manning.  The  re¬ 
sultant  data  are  presented  in  Table  16.  The  data  included  in  Table  16 
parallel  those  found  in  Figures  9  to  11.  Specifically,  in  Table  16,  the 
cumulative  percentage  of  simulations  completed  within  a  given  time 
interval  is  presented  by  malfunction  for  the  simulated  team  composed 
of  two  "fast"  maintenance  technicians  and  for  the  simulated  team  com¬ 
posed  of  two  "slow"  maintenance  technicians.  Again,  malfunction  4 
(replace  receiver  IF  audio  module)  is  indicated  to  be  at  the  extreme 
low  end  of  the  malfunction  correction  time  distribution.  However, 
when  two  technicians  are  assigned  to  the  task,  malfunction  corrections 
11  (replace  noise  blanker  module)  and  24  (replace  interlock  switch) 
also  assume  some  low  ordering.  Malfunction  correction  25  (replace 
RF  board)  is,  as  in  the  series  case,  the  malfunction  with  the  greatest 
predicted  correction  time.  The  range  of  median  values  over  the  10 
simulated  malfunction  corrections  involving  parallel  manning  is  sum¬ 
marized  below: 


"fast"  team 
"average"  team 
"slow"  team 
overall 


-  1315  to  2510  seconds 

-  1335  to  2555  seconds  (interpolated) 

-  1355  to  2590  seconds 

1945  seconds 


Comparison  of  the  above  summary  with  the  prior  summary  sug 
gests  that  the  parallel  manning,  although  increasing  the  mean  probabil 
ity  of  success  over  the  unioperator  manning  from  .74  to  .97,  had  little 
effect  on  the  time  for  successful  malfunction  correction.  Such  a  re¬ 
sult  is  believed  to  be  reasonable  since  the  malfunction  corrections  sim 
ulated  are  of  such  a  nature  that  little  time  savings  can  be  introduced 
by  a  second  man.  Accordingly,  the  present  data  suggest  that  a  second 
man  serves  to  introduce  greater  accuracy  and  to  decrease  the  number 
of  unsuccessful  malfunction  corrections,  but  does  little  in  tei  ms  of 
positively  affecting  the  repair  time. 


Cumulative  Percentage  of  Corrections  within  Stated  Times  for  the  AN/URC-35  (Parallel  Technicians) 
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As  for  the  single  maintenance  technician  case,  there  is  again 
greater  within  team  variation  than  between  team  variation.  The  in¬ 
terpretation  given  in  the  unitechnician  case,  which  ascribed  this  find¬ 
ing  to  the  nature  of  the  AN/URC-35  malfunction  correction  require¬ 
ments,  is  also  offered  here. 

To  summarize  from  the  parallel  manning  point  of  view,  the  tech¬ 
niques  here  applied  yielded  a  predicted  median  malfunction  correction 
time  for  the  AN/ URC-35  which  ranged  between  21.9  minutes  and  43.2 
minutes.  The  best  overall  estimate  of  malfunction  correction  time  in 
this  case  is  32.4  minutes.  The  mean  probability  of  malfunction  cor¬ 
rection  was  estimated  to  be  .  97. 


Series  Case--AN/ APS-115 

The  data  relevant  to  the  time  for  malfunction  correction  in  the 
AN/ APS-115  are  presented  in  the  same  manner  as  were  the  time  pre¬ 
dictions  for  the  AN/URC-35.  Cumulative  percentages  of  successful 
repairs  within  the  given  time  intervals  are  presented,  for  the  five 
AN/ APS- 115  malfunction  corrections  subjected  to  this  analysis,  in  Fig¬ 
ures  12  and  13.  At  least  for  the  malfunction  corrections  sampled,  the 
total  range  of  malfunction  correction  time  in  this  system  seems  to  be 
greater  than  for  the  AN/URC-35.  A  total  range  of  times  from  around 
700  seconds  (11.7  minutesM malfunction  correction  105)  to  4350  seconds 
(72.5  minutesMmalfunction  correction  117)  was  indicated.  Malfunction 
correction  105  involves  replacing  the  AS  synchronizer  module,  while 
malfunction  correction  117  is  concerned  with  the  20  V  power  supply 
m  odule. 

Moreover,  greater  differences  are  noted  between  the  "fast" 
and  the  "slow"  maintenance  technician  in  the  present  case  than  for  the 
AN/URC-35.  The  reason  for  this  finding  seems  to  rest  in  the  increased 
complexity  of  the  AN/APS-115  as  compared  with  the  AN/URC-35. 
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Figure  13.  Predicted  malfunction  correction  time  (single  technician)  for 
malfunctions  114  and  117  (AN/APS- 1 1 5). 


The  range  of  median  malfunction  correction  time  predictions 
for  the  "fast,  "  "average,  "  and  "slow"  maintenance  technician  is: 

"fast"  technician  -  1150  to  3530  seconds 

"average"  technician  -  1213  to  3870  seconds  (interpolated) 

"slow"  technician  -  1275  to  4210  seconds 

overall  -  2541  seconds 

Here,  for  the  single  technician  case,  the  data  suggest  the  median 
time  to  repair  the  AN/ APS- 115  radar  set  to  be  between  19,2  minutes 
and  70.  2  minutes  with  an  overall  median  of  42.  4  minutes.  The  predicted 
probability  of  a  successful  malfunction  correction  within  this  time  inter¬ 
val  was  previously  given  as  .56. 


Parallel  Case--AN/ APS-115 


As  for  the  AN/URC-35  radio  set,  the  data  relative  to  the  parallel 
manning  situation  are  presented  in  tabular  form  (Table  17).  Table  17  in¬ 
dicates  a  rather  large  total  range  (from  975  seconds  to  4375  seconds).  A 
rather  extensive  range  spread  was  also  indicated  for  the  unitechnician 
case.  There  is  also  considerable  difference  indicated,  in  each  case 
sampled,  between  the  predicted  median  malfunction  correction  time  for 
the  "fast"  (more  proficient  team)  and  the  "slow"  (less  proficient)  team. 

As  for  the  unitechnician  malfunction  correction  simulation,  malfunction 
correction  10i(azimuth  servo  amplifier  assembly)  was  predicted  to  take 
the  least  time  and  malfunction  correction  110  (automatic  frequency  con¬ 
trol  assembly)  was  predicted  to  require  the  most  time.  For  all  five 
malfunctions  in  the  sample,  the  range  of  predicted  median  values  is; 


"fast"  team  -  1170  to  3565  seconds 

"average"  team  -  1233  to  3895  seconds  (interpolated) 

"slow"  team  -  1295  to  4225  seconds 

overall  2561  seconds 


Again,  we  note  no  exceptional  time  savings  to  accrue  from  par¬ 
allel  manning.  For  the  AN/ APS- 115  and  for  the  parallel  manning  case, 
the  overall  prediction  of  malfunction  correction  time  is  42.7  minutes. 
The  mean  probability  of  successful  malfunction  completion  is  .91. 
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Table  17 


Cumulative  Percentage  of  Corrections  within  Stated  Times 
_ for  AN/APS-115  (Parallel  Technicians) 


Interval 

(seconds) 

950-1000 

1000-1050 

1050-1100 

1100-1150 

1150-1200 

1200-1250 

1250-1300 

1300-1350 

1350-1400 


2050- 

2100- 

2150- 

2200- 

2250- 

2300- 

2350- 

2400- 

2450- 

2500- 

2550- 

2600- 

2650- 

2700- 

2750- 

2800- 

2850- 

2900- 

2950- 

3000- 

3050- 

3100- 

3150- 

3200- 

3250- 


2100 

2150 

2200 

2250 

2300 

2350 

2400 

2450 

2500 

2550 

2600 

2650 

2700 

2750 

2800 

2850 

2900 

2950 

3000 

3050 

3100 

3150 

3200 

3250 

3300 


_ Mai  function 

101  105 

F*  S  *  F  S 


Correction 

114 

F  S  F 


35 

2 

49 

7 

61 

13 

1 

85 

1 

32 

3 

89 

7 

49 

9 

92 

13 

63 

29 

92 

31 

75 

48 

93 

54 

88 

65 

94 

76 

95 

72 

100 

100 

96 

3 

76 

96 

8 

78 

98 

22 

86 

1 

98 

47 

88 

8 

99 

74 

88 

19 

100 

100 

88 

35 

100 

100 

90 

48 

92 

64 

92 

82 

94 

92 

96 

100 

100 

100 

_  II 

Fast" 

team 

t  f. 

Slow" 

team 

117 

F  S 

Interval 

(seconds) 

110 

F  S 

1 

3350-3400 

5 

1 

3400-3450 

17 

11 

3450-3500 

34 

38 

2 

3500-3550 

47 

68 

6 

3550-3600 

58 

78 

20 

3600-3650 

67 

90 

54 

3650-3700 

74 

93 

74 

3700-3750 

79 

100 

100 

3750-3800 

84 

3800-3850 

86 

3850-3900 

86 

3900-3950 

87 

3950-4000 

88 

1 

4000-4050 

88 

3 

4050-4100 

89 

11 

4100-4150 

89 

26 

4150-4200 

89 

44 

4200-4250 

89 

56 

4250-4300 

90 

78 

4300-4350 

97 

94 

4350-4400 

100 

100 
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Interuser  Reliability 


To  determine  the  extent  of  agreement  among  different  users 
of  the  techniques  for  predicting  technician  reliability,  three  separate 
analysts  performed  job  factor  assignments  to  each  of  the  49  malfunc¬ 
tions  in  the  samples.  Between  analyst  agreement  was  estimated  on 
the  basis  cf  phi  coefficients  and  the  total  agreement  across  the  three  rat¬ 
ers  was  estimated  on  the  basis  of  an  average  phi  coefficient  (Guilford, 
1963,  p.  334). 

Table  18  presents  the  resulting  phi  coefficients.  The  overall  phi 
coefficient  for  the  AN/URC-35  was  .  90  and  for  the  AN/APS-115  it  was 
.  92.  The  agreement  between  the  individual  analysts  also  seems  accept¬ 
able.  The  somewhat  higher  agreement  for  the  AN/APS-115  may  be  at¬ 
tributable  to  the  fact  that  approximately  seven  months  intervened  be¬ 
tween  the  analysis  of  the  AN/URC-35  and  that  of  the  AN/APS-115.  Dur¬ 
ing  this  interval,  the  three  analysts  were  engaged  in  completing  various 
aspects  of  the  program,  and  this  exposure  may  have  contributed  to  a 
greater  understanding  of  the  job  factors. 
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Table  18 


Reliability  Coefficients  Among  Analysts  Using  the  Job  Factors 

URC-35 


Analysts 

A,  B 

JL- 

.94 

A,  C 

.91 

B,  C 

.85 

Overall 

APS-115 

• 

CD 

O 

Analysts 

A,  B 

.89 

A,  C 

• 

CD 

^4 

B,  C 

.89 

Overall 

.92 
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CHAPTER  IV 


DISCUSSION  AND  CONCLUSIONS 


The  major  purpose  of  the  present  study  was  to  appraise  in  a 
Navy  situation  two  techniques  (a  mathematical  technique  for  predict¬ 
ing  probability  of  successful  malfunction  correction  on  the  first  at¬ 
tempt  and  a  computer  simulation  technique  for  predicting  malfunc¬ 
tion  correction  time)  previously  developed  by  Applied  Psychological 
Services  for  assessing  and / or  predicting  technician  reliability  in 
restoring  malfunctioned  equipment.  For  this  purpose,  two  Naval  elec¬ 
tronic  systems  were  used;  Radio  Set  AN/URC-35  and  Radar  Set  AN/ 
APS- 115.  A  sample  of  49  different  malfunctions,  across  the  two  equip¬ 
ments,  was  selected.  The  malfunction  corrections  were  analyzed  in 
terms  of  the  specific  maintenance  job  factors  involved  in  the  correction 
of  the  sampled  malfunctions,  and  the  mathematical  technique  was  applied 
to  yield  malfunction  correction  success  probability  predictions  for  the 
case  in  which  a  single  technician  performs  the  malfunction  correction 
and  for  the  case  in  which  two  technicians  perform  the  repair  actions 
as  a  team.  The  computer  simulation  technique  was  applied  to  a  sub¬ 
sample  of  15  malfunctions  for  the  specific  purpose  of  obtaining  the 
mean  time  (for  both  the  series  and  the  parallel  cases)  to  complete 
satisfactorily  each  malfunction  correction. 

The  data  and  methods  presented  in  the  body  of  this  report  exem¬ 
plify  the  techniques  for  determining  technician  reliability.  The  predic¬ 
tive  validity  of  the  techniques  has  yet  to  be  tested.  Test  of  predictive 
validity  represents  the  next  logical  step  in  the  sequence  of  developing  an 
evaluative  technique  and  will  be  performed  during  the  next  period.  To 
this  end,  those  malfunctions  to  which  both  the  malfunction  correction 
probability  predictive  technique  and  the  computer  simulation  technique 
were  applied  will  be  inserted  into  actual  Fleet  equipments.  A  sample 
of  electronics  technicians  will  be  asked  to  correct  these  malfunctions 
and  performance  measurements  made.  These  performance  measure¬ 
ments  will  be  correlated  with  the  predictive  data  (probability  of  mal¬ 
function  correction  on  the  first  attempt  and  time  for  malfunction  cor¬ 
rection)  here  reported.  Both  the  single  technician  and  the  parallel  tech¬ 
nician  malfunction  correction  cases  will  be  investigated. 


However,  the  validity  of  the  present  set  of  techniques  can  be 
supported  on  other  grounds.  In  regard  to  the  computer  simulation. 
Chapter  I  of  the  present  report  presents  a  number  of  instances  in 
which  the  validity  of  the  Siegel -Wolf  model  was  investigated.  These 
validational  studies  represented  situations  which  were  analogous  to 
those  here  considered.  In  regard  to  the  deterministic  technique,  it 
can  be  argued  that  the  technique  possesses  construct  validity  in  that 
the  job  factors  are  drawn  from  a  factor  analysis  of  the  electronic 
maintenance  job.  Moreover,  the  actual  probabilities  employed  in  the 
calculations  are  derived  from  a  rather  large  Fleet  sample  and,  as 
such,  are  believed  to  represent  an  acceptable  data  bank.  The  reli¬ 
ability  of  these  basic  data  was  demonstrated  by  Siegel  and  Federman 
(1970).  In  that  study,  retest  measures  were  obtained  on  the  evalua¬ 
tions  of  100  technicians.  When  the  means  of  the  eight  ratings  were 
used  and  correlated  with  the  means  of  the  ratings  in  a  second  time 
period,  the  resulting  coefficient  of  correlation  was  .  97.  When  the 
product  moment  coefficient  of  correlation  for  the  individual  technici¬ 
ans  was  calculated,  the  resulting  correlation  was  .72. 

The  probability  combinatorial  methods  employed  to  yield  an 
overall  probability  estimate  are  reasonable  and  are  used  in  other  pre¬ 
dictive  situations.  The  assumptions  of  the  probability  combinatorial 
technique  in  the  present  context  are  not  different  from  those  made 
when  such  probability  combination  is  performed  in  other  contexts. 

The  discriminating  power  of  the  methods  here  described  is 
supported  by  the  indications  that  they  were  sensitive  to  both  system 
and  parametric  variation.  The  techniques  were  able  to  differentiate 
from  both  the  probability  of  successful  malfunction  correction  and  the 
time  for  malfunction  correction  points  of  view  both  across  the  AN/URC- 
35  and  the  AN/ APS- 115  systems  and  for  individual  malfunction  correc¬ 
tions  within  these  systems.  Moreover,  time  differences  were  shown 
by  the  computer  simulation  technique  for  "fast"  (above  average)  and 
for  "slow"  below  average  technicians.  The  lowered  probability  and 
increased  time  estimates  for  the  AN/ APS- 115  as  compared  with  the 
AN/URC-35  might  have  been  predicted  since  the  AN/ APS- 115  is  the 
more  complicated  and  sophisticated  gear  of  the  two. 


Contentions  supporting  the  utility  of  the  techniques  here  involved 
are  supported,  first,  by  the  interanalyst  reliabilities  obtained  in  the 
present  study.  When  three  analysts  separately  performed  the  basic 
analyses  (the  professional  and  training  backgrounds  of  the  analysts  were 
varied  over  psychology  and  mathematics),  phi  coefficients  of  .  90  and 
.  92  were  obtained  for  the  two  electronic  systems  considered.  Second, 
the  methods  were  found  to  be  applicable  to  the  two  different  electronic 
systems  involved  and  to  yield  results  which  appear  to  be  reasonable, 
at  least  from  the  intuitive  point  of  view.  Moreover,  the  reliability 
specialist,  on  the  basis  of  a  set  of  data  such  as  those  here  presented, 
is  able  to  know  which  specific  aspects  o.  a  system  contribute  to  low 
malfunction  correction  probability  or  to  unacceptable  malfunction  cor¬ 
rection  time  predictions.  In  regard  to  the  utility  of  the  techniques,  we 
also  note  that  they  can  be  employed  comparatively  early  in  the  system 
design  cycle,  i.  e. ,  at  a  time  when  design  modification,  if  indicated,  can 
be  implemented. 

We  further  note  that  the  application  of  the  techniques  do  not  de¬ 
pend  on  excessive  mathematical,  electronic,  or  human  factors  sophis¬ 
tication.  They  were  here  employed  by  persons  with  only  reasonable 
backgrounds  in  these  areas.  This  suggests  that  others,  with  similar 
reasonable  backgrounds,  should  be  able  to  employ  the  techniques. 

The  overall  probability  of  malfunction  correction  on  the  first 
attempt  metric  produced  by  the  present  technique  is  highly  similar  in 
characteristics  with  the  equipment  reliability  metric  determined  dur¬ 
ing  the  system  design  stage  by  the  reliability  engineer.  Accordingly, 
it  seems  entirely  reasonable  to  combine  the  current  technician  reli¬ 
ability  index  with  the  equipment  reliability  index  to  yield  a  total  main¬ 
tenance  system  (man  and  equipment)  reliability  index. 

Finally,  and  of  some  importance,  the  computer  simulation  logic 
has  been  tailored,  through  the  preprocessing  subroutine,  to  be  entirely 
compatible  with  the  probabilistic,  combinatorial  technique.  The  two 
methods  constitute  a  complementary  set.  While  the  data  yielded  by 
each  method  are  different,  the  data  produced  by  each  method  are  en¬ 
tirely  interpretable  against  the  backdrop  of  the  data  yielded  by  the 
other  method.  Hence,  the  two  techniques  may  be  held  to  represent 
a  coordinated  set  which,  when  applied,  yields  an  integrated  statement 
regarding  maintenance  technician  reliability  in  the  context  of  a  specific 
equipment  system. 
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Conclusions 


The  following  conclusions  seem  indicated: 

1.  The  utility,  applicability,  discriminating  power, 
and  reasonableness  of  the  mathematical,  proba- 
balistic,  job  factor  combinatorial  method  for  pre¬ 
dicting  probability  of  successful  malfunction  cor¬ 
rection  on  the  first  attempt  appear  to  have  been 
demonstrated. 

2.  Similarly,  the  computer  simulation  method  for  pre¬ 
dicting  malfunction  correction  time  appears  to  repre¬ 
sent  a  reasonable  method  for  achieving  this  end  result. 

3.  The  two  techniques  may  be  used  individually  or  in  con¬ 
cert  in  accordance  with  the  data  and  information  require¬ 
ments  of  an  individual  system  developmental  program. 

4.  The  predictive  validity  of  the  techniques  remain  to  be 
demonstrated  in  a  malfunction  repair  context. 
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APPENDIX  A 


Froject  RAM  Computer  Data  Element  List 


FORTRAN  Name 
SPOG(JA,  JT) 
JAG(IJ) 

NG 

PG(  JA,  J) 

TP(JA,  T) 
ICE(JA,  J) 
TNR(JA,J) 
AID(JA,  J) 

JT 

JOT 

JA 

OLPROB(IJ) 

J 

I 


_ Meaning/ Function 

SUCCESS  PROBABILITY  OF  GROUP 

JOB  ACTIVITY  GROUP 

NO  OF  TASK  ELEMENTS  IN  GROUP 

PRODUCT  OF  PROBABILITIES  IN  GROUP 

TOTAL  POTENTIAL  INCREASE 

ITS  CLOSE  ENOUGH  INDICATOR 

(NG)THE  ROOT  OF  PG 

AVERAGE  INCREASE/ DECREASE 

OPERATOR  TYPE 

JOB  ACTIVITY  TYPE 

JOB  ACTIVITY 

STORAGE  FOR  ORIGINAL  VALUES  OF  TASK 
ELEMENT  SUCCESS  PROBABILITIES 

OPERATOR  NUMBER 

TASK  ELEMENT  NO 


Preceding  page  blank 
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